Off-box logging is vital to everyone's sanity and success when in AWS.
Using Papertrail has been painless and the features are great! There are a many different ways to integrate with them, and forwarding your syslog to them is a good first step.
The problem I ran into was that my logs were showing up by hostname. In AWS, a hostname of "ip-nnn-nnn-nnn-nnn" is not very clear. What application does that instance belong to? Is that instance part of staging or production? Add another complication - AutoScaling. Now when an instance comes online and goes away after a few hours, it gets even more difficult to understand what is occurring when you look at the logs.
In my UserData, in the CloudFormation Template, I just added these simple bash lines to change the hostname to something meaningful and dynamic:
The CloudFormation Template has two input parameters (AppName and Environment) and uses a Join to put the hostname variable together.
So now the hostname will be <ApplicationName>-<Environment>/<InstanceID>.
Back in Papertrail, I can now group these systems using wildcards.