This blog has moved
Sunday, March 22, 2020
Moving on...
Thanks for visiting this blog.
After having this blog for more than 13 years, I have decided to retire it. My new blog can be found at rohitsm.com hosted on Github Pages (rohitsm.github.io is the URL to be precise). All the content hosted here will continue to remain accessible at the URL oldblog.rohitsm.com.
Thursday, July 4, 2019
Serializing Spark dataframes to Avro using KafkaAvroSerializer
I recently worked on a project that used Spark Structured Streaming using Apache Spark, Confluent Schema Registry and Apache Kafka. Due to some versioning constraints between the various components, I had to write a custom implementation of the KafkaAvroSerializer class for serializing Spark Dataframes into Avro format. The serialized data was then published to Kafka. This post is based on the examples specified in the Confluent documentation here.
In newer versions of Confluent Schema Registry, lot of the implementations detailed below have been simplified and much easier to use. The standard recommended usage of the Confluent KafkaAvroSerializer is fairly simple in that it requires you to set it as one of the Kafka properties that is used when initializing a KafkaProducer:
props.put(...)
...
...
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, io.confluent.kafka.serializers.KafkaAvroSerializer.class
val producer = new KafkaProducer(props);
Thursday, January 17, 2019
My 2018 Reading Challenge
Bad Blood by John Carreyrou
Sunday, August 14, 2016
Book Review: The Intel Trinity: How Robert Noyce, Gordon Moore, and Andy Grove Built the World's Most Important Company
The Intel Trinity: How Robert Noyce, Gordon Moore, and Andy Grove Built the World's Most Important Company by Michael S. MaloneMy rating: 5 of 5 stars
I began reading this book on a whim having already been familiar with the backgrounds of the three founders - Robert Noyce, Gordon Moore and Andy Grove. Michael Malone chronicles the birth of Intel to its present state methodically and brilliantly. During the course of the narrative, the book beautifully digresses to give the reader a detailed look into the early lives of the three founders starting with Noyce, Moore and finally Grove.
The book acknowledges that it borrows from the individual biographies of its founders but doesn't come across as redundant at any point. Deeply engaging in its narrative and often poignant in its story line, I found this book beautifully compiled and enjoyed every bit of it to the point that I was a little sad when it ended.
If you are a fan of biographies, you would find this to be a great book. I'd rate is So-good-I-can't-stop-telling-you-about-it good!
Sunday, November 2, 2014
#Humblebrag - This blog got featured in a documentary on Aaron Swartz
How do I know its mine? The novice level of the crudely written code is uniquely me. I was supposed to improve upon it later but I never really got around to actually doing it. Meanwhile some people who forked the code from my Github repo did a far better job anyway. This project was more of a tribute than anything else and so it was a humbling experience to have randomly spotted it while watching the documentary again yesterday.
Tuesday, July 1, 2014
30 Day Challenge - July 2014 and a brief summary
Here are the list of 30 day challenges that I've attempted in the past. I've had fun doing some of them and have blogged about my experiences.
Thursday, January 2, 2014
Report on 30 Day Challenge - Read 15 books in 30 days
This post has been long overdue but considering that I did not attempt to complete the remaining four books until now, I guess this post is not so ill-timed after all. I should mention here that my previous (best) failed attempt at this same challenge was 12 books. I am hoping to beat this sometime in future but that would depend on whether, and when, I would get my hands on a stack of books this engaging.
Friday, January 25, 2013
Remembering Aaron Swartz - Raw Thought
Be curious; Read widely. Try new things. I think a lot of what people call intelligence just boils down to curiosity.- Aaron Swartz
![]() |
| Aaron Swartz (1986 - 2013) |
Thursday, December 27, 2012
Happy Holidays!
Saturday, August 18, 2012
Getting Flash to run on Tor for Mac
Tor, by default does not support third party Firefox plugins such as Flash and Javascript and this makes it difficult to view websites such as YouTube and other sites that require flash. This tutorial aims to address the issue for Tor Browser running on Mac OS X.
We already assume that you have Tor browser installed and running. If you don't have it installed, you can download it from here and follow the installation instructions.
Step 1: Download and install Flash Player for Mac.
Friday, March 16, 2012
Anatomy of a busy month
Wednesday, November 23, 2011
Interview for IES
I wanted to share this now that its up on their website.

Tuesday, November 1, 2011
30 Day Challenge for October
Sunday, October 2, 2011
Report on 30 Day Challenge for September
This challenge might seem a little mundane but the idea was to break out of this habit. I am glad I did. Its been more than a month now and I still haven't really felt the urge to pop a cold one.
Wednesday, August 10, 2011
In conversation with Tan Jie Yim Priscilla
“Stilt walking is not an easy task and it needs balancing which is most important” says Priscilla who has been performing since 2004. “When I started off, it was very scary and I took a few hours to learn and balance myself fully”. Standing at a height of two meters, seeing Priscilla perform often leaves viewers in awe in addition to giving them goosebumps. “After you tie everything up and are ready to take the first step, you are always wary of falling and keep wanting to hold something” When asked if she had any bad experiences while performing, Priscilla mentions a time when she fell during one of her performances after accidentally stepping on a patch of water and some flowers. “The key is to fall forward with your knees bent, instead of backwards, as the shin guards and hand pumps offer protection and support.” She mentions determination and discipline towards practice sessions as vital traits of a stilt walker’s success.
She acknowledges her family and friends who supported her during her studies and encouraged her to work hard towards her aspirations. “I had a hard time coping with my studies, but I always had my family, especially my Dad and boyfriend, whose constant encouragement kept me going” recalls Priscilla who now works as an Admin Executive at Trillion Lifestyle Pte. Ltd. She hopes to work hard and eventually tour the world with her parents who, despite setbacks, kept going and never gave up.
With her hardwork and persistence, we are certain that she will reach even greater heights in life - those that lie beyond the realms of stilt walking.
Friday, June 3, 2011
10 years ago on this day...
| The beautiful hillside of Panchgani |
![]() |
| A few of us at our farewell dinner - March 2005 |
Tuesday, May 17, 2011
30 Day Challenge for May
My 30 day challenge for May is to "track and keep my finances in order". I intend to note down my daily expenses, tabulate it and keep a record of it for the next 30 days. With this I hope to get an overview of all my expenses and see if there is any particular area that needs looking into. I've been trying to do this task for quite a while but I didn't really focus on it. But now I am determined to do it as a challenge. Of course, having an iPhone does help an awful lot. I am using the Money Journal v1.4 (Lite) app to help me record my daily expenses.
Even though this post is dated 17 May 2011, I have already begun this challenge since the 1st of May. I shall report on my progress within a fortnight.
Saturday, April 2, 2011
30 Day Challenge for February
My 30 Day Challenge for February was 'to walk home daily from college'. Since February 2011 has only 28 days, I began this challenge on the 31st of January and extended it to the 1st of March. This challenge would have been a 20 day challenge because theres no school on Saturdays and Sundays, but I did my walking on weekends too and completed all 30 days.
The distance from my school to my block is about 2 miles which roughly takes me about 35mins to cover (Approx speed = 3.5mph). With the help of a walking calorie calculator I was found out that I burned an average of 186 calories per day. This means that in 30 days, I burned roughly 5580 calories by walking home every evening. For me, the benefits did not end there. A typical one way trip home from school by bus costs me $0.81. Over a period of 30 days, I ended up saving $24. All it took was a little music and some podcasts that I follow quite regularly.
I did not attempt this challenge with the aim of saving money or losing weight (I play soccer regularly and my BMI is within the 'healthy' range). It was mainly aimed at cultivating some self discipline and see if I could keep up with it for 30 days. It was hard though, mainly on weekends when I had no particular reason to step out from home. 30 days later, I have eventually liked walking, particularly in the evenings, because it gives me time for myself, to reflect on the day's events and the pleasant weather is a bonus. All in all, I am happy with the way this challenge had progressed and hope to continue walking home in the days to come.
Wednesday, February 23, 2011
Fireside chat with Ms. Wang Yuyan
Back in 2006, TP introduced its new Diploma in Integrated Facilities Management (IFM) course. Five years on, it has become one of the most sought-after courses in TP. We caught up with Ms. Wang Yuyan, one of the students from the pilot batch of 2006, and who graduated in 2009, to see how she was coping with life after TP.
Ms. Wang recently graduated from the prestigious International Hotel Management Institute (IMI) in Lucrene, Switzerland with a BA (Hons.) in International Hotel and Tourism Management. It is interesting to note that Ms. Wang’s acceptance into IMI throws light on how versatile and well recognized TP’s Diploma in IFM is. Taking time out to answer a questionnaire, she credits TP for preparing her for the future. “My Diploma in IFM provided me with the basic foundation for the hospitality industry. It has helped me to better understand what was being taught in class and as a diploma graduate it allowed me to advance to a higher level of study” she remembers. Some of her favourite subjects included Introduction to Hospitality and Tourism, Introduction to Language and Culture (French) and Attractions Management and Club and Resort Business. She fondly recalled memories of staying late in school with friends to meet project deadlines and how much the facilities provided by TP helped her to conduct her own self- study or for doing group work with friends.
Talking about life in Switzerland, she described how much she thoroughly enjoyed studying in Europe. “In our free time, my classmates and I would take the opportunity to travel around Europe. So far I have been to Milan, London and Amsterdam, places I would never think I'd visit. It has been quite an experience so far as it is my first time studying abroad and I am enjoying every single bit of it”.
Ms. Wang aspires to hold a managerial position at a five star hotel someday. We wish her all
the very best in this endeavor.
Saturday, February 19, 2011
30 Day Challenge for January
I will be travelling in March and I intend to redo this challenge, this time with better focus. As for the photos that I took this past month, here they are*.
I managed to take only 17 from my target of 30. Some of the pictures taken at night are grainy too. Apologies. I shall remember to carry, with me, a better camera next time.
Till next time...
*Update: When Blogger was updated to the newer version, my embedded slideshow broke as a result. So I had to add a link to the G+ album instead.




