The tools of our trade

Oct 05, 2017

Wouldn't it be awesome if computers could do most of your work? We believe that for one to be productive in one's work, distractions need to fade into the background. Distractions like monitoring progress of team mates, looking after the health of computers in the lab and taking regular backups of code and data are easily avoidable by employing some really cool technologies such as the ones I'm about to describe. Maybe you'll find some nice things to learn and maybe you'll be able to put some of these to use yourself! So hang tight and read on.

Git - version control

Believe it or not, writing code in a team leads to a lot of conflicts. Things go awry, the code breaks down and all sorts of problems start popping up. People end up overwriting each others' code, people end up deleting each others' code and people end up overriding each others' code. These sorts of wreckage occurring because of human error can easily be avoiding by removing the human from the equation.

Version control is the solution to this. Version control software can help you keep track of the evolution of your code, allowing you to reset back to any state in its lifetime. Version control also allows many developers to work on the same code (on different branches) at the same time and then merge their progress into the main codebase. Version control also solves the problem of communication by allowing each and every member to keep a track of other people's progress on their tasks.

Enter Git, the fully distributed-centralized-remote-local-branched-fast version-control source-code-management tool. Git is the quintessential developer tool. It is one of the most, if not the most, important tools in a developer's toolkit. You can't really become a developer unless you master the art of version control and Git is without a doubt the favourite one out there. 

Git is centralized as well as distributed, meaning that though there is a central repository regarded as the origin, each and every user has a complete clone of the origin repo and should the central server fail, any other up to date clone can be re-designated as the new origin. For the central server management, we use GitLab, our in-house Git server for security and privacy reasons, but you can start off with GitHub or Atlassian Bitbucket, which will handle the hosting for you for free!

Sentry - bug tracking

Code written by mere mortal developers inevitably ships with bugs, regardless of how deeply it was tested and how many iterations it has gone through. Bugs are a part and parcel of development and it is better to embrace and confront them rather than shy away from admitting it. But how do we get notified should a bug reveal itself in the wild? Expecting a user facing a bug to submit a full fledged bug report form is nonsense. No one likes to fill a form explaining how things went awry especially just after a lot of their expectations from the product have been shattered and their efforts have gone to dust. 

Sentry is awesome for the simple reason that it helps us find our bugs without having to rely on user feedback, which is rare and very misleading. Sentry shows us our error logs, stack traces and the state of the system when the bug occurred, largely taking away the need to reproduce the bug and increasing the rate at which we identify, locate and fix our bugs. In fact, we use Sentry statistics in our fortnightly meetings to identify our most active bug-fixers!

Just like Sentry on production, we run a bug reporting service for our development servers. It's called Bugzilla. Bugzilla takes bug reports from willing reporters on our own team. But because of Bugzilla's dated UI and bloat, we are planning to shift this reporting and tracking system to our in-house developed product Parche, which has a more sound technical stack and a fresh UI.

Status - health monitoring

Infrastructure is as interesting as civil engineering. Yeah, not interesting at all. But it is also as important as civil engineering. Very. As you might be aware, one application does not run on one server. It is made up of several microservices spanning multiple computers across multiple networks. And if one of these microservices breaks down, it takes the entire application down with it. Understandably, keeping the infrastructure in good health is a persistent priority.

But we cannot allow this priority to become an omnipresent headache and hamper our productivity by distracting us away from the task at hand and towards running health tests. So we automated it. Status is IMG's very own system health monitoring tool, which helps us keep track of the health of our servers, and the status of our services running on various machines. Status keeps logs of the various uptimes and downtimes and gives us up-to-the-second information about the state of our infrastructure, the importance of which we cannot overstate. For a lone wolf developer, Status is not that very useful but for an organization or a group, Status is a mission-critical must-have.

Status pings various IPs and collects system logs from the services running on those machines. It automatically notifies us if any of our services don't work as expected and we frequently go through Status' logs to ensure that everything is running smoothly.

Piwik - analytics

Analytics are supremely important. When thousands of users rely on your application, you need to be able to take decisions backed by quantifiable statistics. Let's say we suddenly roll out a large scale change to the popular app LecTut. How will that affect professors who take time to adjust to new technology? How will it affect students who are always eager to see UI changes? How will it affect the way people use the application? Let's say we add a new feature to an app. How many people will notice it? How can we increase the discoverability of the feature? How can we ensure that the existing features are not disrupted by it? Let's say we roll out a new application altogether. How many people will take interest in it? How many will use it initially? How many after a week? After a month?

All these questions can either haunt you every time you sit down to write some code, or you could use analytics and solve this once and for all. Piwik provides us the customized analytics we need in addition to the Google Analytics which is, understandably, already set up on our sites. We use Google Analytics for the bird's eye view of the applications, their engagement and usage statistics. Analytics tells us about the platform used by our users, their clicks on various pages and views on the apps and the site as a whole. We use Piwik for the more detailed minute statistics that we can customize according to our requirements.

So when it comes to extrapolating user activity to meaningful statistics, Big Brother is always watching.

Wiki - documentation

People usually loathe documenting their code. But then they return to it after a month or an year and are unable to comprehend their own code. This is something we cannot afford because at IMG members come and leave and the code they wrote remains. The person looking after an application is rarely ever the same person who wrote it. We cannot expect a person to go through the entire code should a problem arise.

This is where documentation reveals its utility. Well documented code results in lesser pain for the maintainer and improves the efficiency of the group. But just as important as writing good documentation is writing documentation in the right way. Storing folders upon folders of text documents on Google Drives is not the way to document. Keeping comments scattered around the code is not a good idea either. Having a centralized repository of documentation where everyone can drop in their app documentation for anyone else to read is something I will willingly get behind. At IMG, that repository is the IMG Wiki.

At IMG we have a MediaWiki setup so that anyone and everyone who is experienced in any field of development can contribute to this compendium of knowledge and can ensure that after his time in the institute is over someone else will be able to take up his mantle and continue the work he has done. At IMG, the knowledge of our ancestors is never forgotten or lost, rather it is refined, grown and then passed down from generation to generation.

Slack - communication

Communication is essential to a team's success. Whether or not you agree, we have observed that things go awry when teams don't communicated well. And when it comes to keeping the entire team in the loop across the spectrum of activities running in the team, messaging services like Facebook and WhatsApp simply don't cut it. We need something specialized, something refined. We found Slack.

Slack is the communication tool for teams. It helps the entire work of the group to be divided into channels and allows us to carry on many coversations on many topics at the same time. We have channels for various apps, a channel for bugs, a channel for bakar and a channel for various events. Each channel runs a separate conversation and so we can stay updated on everything happening in the group.

Trello - task management

Amidst all these tooling, we cannot afford to miss out on the To-do list, which is the ultra-basic but ultra-essential friend to every developer. Again, keeping our tasks organized is important, no doubt, but as a team it is important to stay updated with the to-do list of our friends as well.

Trello is a to-do list but for teams. It divides the entire bunch of stuff that is to be done and divides it into boards, further dividing it into sections and finally into cards. Then you can assign deadlines and update progress, upload attachments and the like. In the end, you have a full fledged dashboard where you can learn about the stage of every single task undertaken by the group and its various members.

Closing remarks

I sincerely hope that gives some more clarity into the way we handle our tasks and make amazing applications for the campus at astonishing rates. If you found something interesting and are taking something from this article, do let us know. And as always, I'll see you in the next blog post.