Learning by Doing – An AWS Project

Like in many jobs it is not easy for mine to squeeze in the regular learning to stay on top of new technologies, techniques, process etc. It’s too easy to get busy with the seemingly urgent topics of the day.

There are three strategies that work best for me to keep my self up to date:

  1. Get half an hour learning in before you start your workday. I try to do that every day during breakfast before I even start up my work computer. Once I start my computer it’s over, I get pulled into emails or even worse: slack conversations. As 30 minutes is not a lot of time I usually spent it reading a technology book. And yes, sometimes it is not even 30minutes because the baby wants to be fed or the lawn needs to be watered, but every minute counts. Even if you spend just 10minutes reading every day, it is more than an hour of reading every week.
  2. Go to conferences. Build a conference calendar for your year. Conferences are great for both getting high-level and in-depth knowledge about technologies and they give you the opportunity to meet other users and experts in the field.
  3. Come up with a small project that allows you to use some of those new things you want to learn. I don’t know about you but I learn best by doing stuff. So sometimes I look at my day and try to think which thing that I’m doing I can solve with a completely over-engineered solution merely for the purpose of learning something 🙂

This article is about one of those projects.

The „problem“

I’m thinking of selling my current car next year and instead of buying a new one lease a one instead. One of the parameters that affect the cost of leasing is the distance you plan to drive every year for the duration of the contract. My guess is it will be a bit less than 25000km, but I’d like to know for sure as I don’t want pay more than needed but also don’t want to drive more than what is allowed in the leasing contract.

The non-technology solution that I’m using right now and which works perfectly fine consists of a pencil and a post-it note:

Foto 19.07.18, 10 39 11

Every once in a while I write down the date and the total distance in kilometers (the German word on top says „mileage“). If I do this long enough I should have a pretty good idea what amount of kilometers to put into the leasing contract.

The over-engineered solution

Wouldn’t it be nice if I simply could take a picture and the mileage got recorded somewhere on the internet? Yes, of course, that would be nice and also I can do this all with AWS and a bit of Golang and Dropbox.

So, here is what I’m planning to do:

dashboard analyser

When I’m sitting in my car, I will take a picture of the dashboard which includes the mileage.

Foto 09.07.18, 11 26 04

I’ll take that picture and upload it to Dropbox. Dropbox will shoot a message to AWS API Gateway that something changed on the monitored folder. API GW triggers a Lambda function that merely publishes a message to a topic on AWS Simple Notification Service.

Another Lambda function subscribes to that topic. When it gets triggered it calls Dropbox to get the latest files that got created, saves them to AWS S3 and deletes them on Dropbox. It also saves a Dropbox cursor to a table in Dynamo DB. A cursor is a pointer to a certain state of the folder when you last looked at it. So if you pass that cursor into a  future list_folder call of Dropbox’s API you get the files that changed since when you called that function the last time. Lambda gets the Dropbox API key from AWS Secretsmanager.

When the image is stored on S3 a third Lambda function is called which utilizes the AWS Rekognition Service to extract the mileage from the dashboard.  After extracting the data it saves it to a different table in DynamoDB and deletes the file on S3.

Eventually, I want to build a simple UI that allows me to view the data somewhat graphically. For that, I’ll utilize all the same service that I used above.

ui design

S3 will serve a static webpage with some Javascript which will asynchronously call API Gateway. API GW triggers a Lambda function that simply accesses the data in DynamoDB that we previously saved there.

So far I have the basic AWS infrastructure set up and the first Lambda function is done. Now I’m working on the second one and then finally it gets to the most interesting piece of use the Rekognition API.

Over the next couple of weeks, I likely post some details of this project. I hope you’ll find this interesting!