API metrics with a single log line

Have you ever wondered, how can I build just a “simple dashboard” to save some metrics without much trouble? Let me tell you, all the code you need:

console.log(JSON.stringify(yourApiResponseHeaders));

That’s pretty much it. In my case, I was using AWS Lambda which automatically stores logs generated by my code through Amazon CloudWatch Logs. Just make sure that your logs are being sent to AWS.

For eg, you can see below the output structure of my logs:

{‘GreatHeader_X_Rate_Limit’: ‘300’, ‘GreatHeader_X_Rate_Remaining’: ‘292’, ‘GreatHeader_X_Rate_Consumed’: ‘8’}

Now, hop into AWS Cloudwatch > dashboards (create one if you don’t have yet) > new widget > graph line or whatever fits best for you

Then, it’s where the magic comes in. Not only AWS provides a great CWL query syntax to analyse logs as it also will parse logs automatically if they are in json format (until a certain limit).

This is what worked for me:

fields @timestamp, @requestId,
`GreatHeader_X_Rate_Consumed` as `Consumed`, 
`GreatHeader_X_Rate_Remaining` as `Remaining` |
filter ispresent(`GreatHeader_X_Rate_Consumed`)

Just hit run query and Bamm! An extensive table with all your logs.

Feel free to add any other field missing. AWS recognises automatically up to 100 fields if they are properly logged.

Now you can easily tweak your query according to your needs. Just check CWL query syntax or some examples here. For me, in order to build a graph grouped by minute, I had to slightly update the query:

fields @timestamp, @requestId,
`GreatHeader_X_Rate_Consumed` as `Consumed`, 
`GreatHeader_X_Rate_Remaining` as `Remaining` |
filter ispresent(`GreatHeader_X_Rate_Consumed`) |
stats sum(`Consumed`) by bin(60s)

At the end, just hit “Create Widget” and there you have it! A “simple” dashboard to track your metrics. Also, don’t forget to check the pricing in order to avoid unnecessary surprises.