Blog: My first month as a Data Engineer Trainee
I am Paul and in this blog we dive into my first month as a Data Engineer Trainee. It is a month with a lot of firsts. My first grown men’s job, the first big data engineering trainee class, my first well-deserved Friday drinks, and last but not least I got my first Xomnia Hoodie.
At Xomnia I joined the traineeship for Big Data Engineering. My first month as a Data Engineer is awesome. Four days a week I work for a client and one day I have a day full of technical training and personal development. The combination of a training and working for a client really help you boost your big data capabilities, I will tell you more about this in this Blog.
Why did I choose the Big Data Engineering track?
Forrester wrote a report that ‘Data Engineers Have Become More Important Than Data Scientists’, and sorry my fellow Data Science colleagues I believe in it. As a Data Engineer we construct and maintain the Real-time Big data lake. A Data lake is a large storage place that stores all the data of the company in its original format until information is required to solve a business need. I love the fact that as a Big Data Engineer you can make the difference for the Data Scientist. We make sure that we create a data flow between different information systems, to provide the Data Scientists good data quality to work with. As a Big Data Engineer I communicate a lot with the Data Scientist, to ensure that the real-time Big data lake supports the requirements, and business needs can actually be solved with data.
Working as Data Engineer at KLM Royal Dutch Airlines
I work for the client KLM Royal Dutch Airlines (short: KLM). Here I support two teams. One team is building a new real-time Big data lake and the other team is supporting the operational decision-making within KLM. The subjects are really varied, from working on the boarding process till keeping track of the aircraft maintenance. At KLM we use various innovative Apache services, such as Hadoop, Hive, and Spark. Most of the code is written in Scala and Python.
It is fantastic that I have the opportunity to expand my skill set. Already in my second week I learned about Apache Hive, which I could immediately use the experience I gained. I helped KLM with finding new ways to load the data from Hive. This needed to happen in such a way that the data is accessible as quickly as possible and that it isn’t locked by the Hadoop Distributed File System. How cool is that? When you learn something new and can use it for solutions in your daily work. It’s awesome to work on a big data platform that helps KLM to really improve their business processes.
Xomnia Friday = soaking up knowledge day
Xomnia Friday passes by with the wink of an eye. The day is filled with interesting opportunities and a lot of laughing with the other trainees. The trainees at Xomnia have various backgrounds. People have backgrounds in mathematics, econometrics, computer science, data science, and artificial intelligence. I have a background in cloud computing, this is a specialization within the field of Computer Science.
After my first month, I can truly say that Xomnia is My Big Data Partner.
Big Data Engineer Trainee at Xomnia and KLM