Weekly Links List – Hadoop and Spark usage at Yahoo; Increasing impact beyond core technical skills; Robotics for physical rehabilitation; Vendelligence initial build

Article Link

Source: Peter Cnudde on How Yahoo Uses Hadoop, Deep Learning and Big Data Platform

Why listed? During much of the tutorials and reading on Apache Spark, they regularly refer back to Apache Hadoop, which I had not personally used, although was familiar with the system as I just started experimenting with Apache Spark last year and wanted to understand the history behind it. I always find it interesting when folks from the larger companies with huge data volumes talk about their use cases and experiences with various technologies, so this is a good interview to hear how Yahoo are using Hadoop, given the history of Hadoop adoption there, and where they specifically target Spark usage.

Article Link

Source: How to Grow from Being Average to a 10x Engineer

Why listed? I was a fan of The Effective Engineer book when it came out last year. Edmund focuses on stuff here that often gets overlooked or laughed at by people because it is not seen as purely technical. Too often, folks focus on technical skills only and not problem solving in general or communication skills as these are seen as “soft skills” and not important. From my own experience, clients and stakeholders expect both and the best engineers I worked with were eager to speak on calls directly or get onsite with the clients, not just only do code or only do talking. This article touches on key points like prioritising the task that has more impact for a user, communications with your internal team or stakeholders, and conflict resolution. I have to remind myself always to try and practice much of what Edmund talks about on his blog regardless of experience. His newsletter is worth subscribing to if you do not have a copy of the book.

Article Link

Source: Soft robots that mimic human muscles

Why listed? This is a short piece, but it is always inspiring to see biomedical research work. These students and researchers have taken relatively cheap materials and adapted them to make robots that mimic muscles and are exploring other use cases to help with physical rehabilitation and patients with impaired motor sensitivity. Has an embedded short YouTube video that demonstrates the various prototypes.

Tool Mention

Tool: Vendelligence

Functions used: Research, Filtering

Why listed? Vendelligence is an open source project I have been working for some time, and after packing in my company activity last month and returning to part-time study since September, I decided to open source a community edition as it allows others to clone, use, or adapt something I use myself for my own tasks. It is a very simple Spring Boot application using Google APIs. I will upload the initial build to GitHub over the weekend, and next week will write a blog on informazine.com, so you have some context on why I built it, provide a high-level solution diagram, so you can see the modules I will focus on myself, and create initial supporting materials for people who want to clone it for customisation or wish to use the community shared repository. For folks who have better things to do than play with the source build, I will create a WordPress portal directory, so you can still use tool functionality directly without running any of the source or hosting your own server.

Leave a Reply

Your email address will not be published. Required fields are marked *