Wed Feb 14 2018

Blog: My first month as a junior data engineer

HEY YOU!

I am Paul and in this blog, we dive into my first month as a Junior Data Engineer. It was a month with a lot of firsts. My first grown men’s job, the first junior data engineering class, my first well-deserved Friday drinks, and last but not least I  got my first Xomnia Hoodie.

At Xomnia I joined the Junior Development program for Data Engineering. My first month as a Data  Engineer was awesome. Four days a week I worked for a client and one day (Friday) I had a day full of technical training and personal development. The combination of trainings and working for a client really helps you to boost your data capabilities, and I will tell you more about this in this Blog.

Why did I choose the Junior Data Engineering track?

Forrester wrote a report that ‘Data Engineers Have Become More Important Than Data Scientists’, and sorry my fellow Data Science colleagues, I believe in it. As a Data Engineer, we construct and maintain the Real-time Big data lake. A Data lake is a large storage place that stores all the data of the company in its original format until the information is required to solve a business need. I love the fact that as a Data Engineer you can make the difference for the Data Scientist. We make sure that we create a data flow between different information systems to provide the Data  Scientists with good data quality to work with. As a Data Engineer I communicate a lot with the Data Scientist, to ensure that the real-time Big data lake supports the requirements, and business needs can actually be solved with data.

Working as a Data Engineer at KLM Royal Dutch Airlines

I worked for the client KLM Royal Dutch Airlines (short: KLM) where I supported two teams. One team was  building a new real-time Big data lake while the other team was supporting the operational decision-making within KLM. The subjects were  really varied, from working on the boarding process until keeping track of the aircraft maintenance. At KLM we used various innovative Apache services, such as Hadoop, Hive, and Spark. Most of the code was written in Scala and Python.

It was fantastic to have the opportunity to expand my skill set at the client. Already in my second week of training at Xomnia, I learned about Apache Hive and I was able to immediately apply it at my client. I helped KLM with finding new ways to load the data from Hive so the data could be accessed as quickly as possible and wouldn’t be locked by the Hadoop Distributed File System. How cool is that, don’t you think? learning something new that I could use for solutions in my work activities. It was awesome to work on a big data platform that helped KLM to really improve their business processes!

Xomnia Friday = soaking up knowledge day

Xomnia Friday passes by in the blink of an eye. The day is filled with interesting opportunities and a lot of laughter with the other Juniors. The Juniors at Xomnia have various backgrounds: People come from mathematics, econometrics, computer science, data science, and artificial intelligence. I, for example, have a background in cloud computing, which is a specialization within the field of Computer Science.

After my first month, I can say that Xomnia is truly having an impact on both artificial intelligence and the Juniors that are part of its team.

Paul Velthuis

Junior Data Engineer at Xomnia and KLM