We have a lot of people meticulously draft feature requests and raise them on the repository's issues page. After a feature request gets reviewed by maintainers and labeled available, contributors get creative and open Pull Requests against these requests with a working implementation.
Most of the time, it can get a little tempting for me to get started on some of the issues right away if I'm sure I know how to implement it or sometimes to try is just plain fun.
However, that's not always the case because there can be times when I simply don't know how to do something. That's when contributors help by raising Pull Requests, and that's the beauty of open source. I get to work alongside people who have experience in certain things that I don't necessarily have and they help fill this gap.
The feature request
A user raised the timeline feature quite some time back. It wasn't actively worked on because the implementation potentially introduced unnecessary complexity to the codebase since data handling existing at that time was convoluted. Things could've been a lot easier if data handling was done a little differently, which I'll be addressing below.
Everything but the timeline mentioned in the description was already present on covid19india.org.
Only the timeline that one can use to scrub through days and visualize the increase/decrease in the intensity of cases over time needed work.
Investigating the current structure
So now, I know that we need the timeline to function on the home page.
Before I start working on it, I revisit the code and open up the source code for the home component. I look around and remind myself how the component is currently structured and how the data flow takes place.
For the sake of simplicity, I'll be showing them as pictures instead of code.
I realize that three significant components show the statistics on the homepage. All of them currently show the current day's statistics, so using a timeline, people can choose a date and view the statistics for that specific day.
Now I ask myself whether the data to go back in time and refer to past statistics was readily available from the APIs we use, and the answer was a little complicated.
The Timeseries component contains the graphs that already have past statistics available because, of course, they require past statistics to plot the graphs. Then, one can think that this time-series data could then be used in the Table and Map Explorer components as well, right?
The addition of new data points was carried out iteratively during the initial weeks of development on both the website and the API. First, we started by putting out only state-wise data, then moved to statewise timeseries, and then the rest. As a result, the endpoints were added to the website one by one as they came in from the API, and we had to route it to the appropriate components by mixing and matching data points here and there.
Shoutout to @JunaidBabu for rapidly pushing these endpoints out
For instance, the home component used four endpoints to fetch all the required data needed to be displayed on the home page.
All of these then get parsed on arrival and supplied as props to each of the components.
On the other hand, timeseries data had state-level information only, so we couldn't use that in the table that had district-level statistics as well unless we programmed an explicit fallback mechanism. The props to the table and rows were structured differently and wouldn't work with the timeseries data without little modifications to the frontend.
So now we had to support district-level data in timeseries format. That meant we would be adding ~735 districts x 4 statistics x 90 days so far — a whopping 4.2MB worth of data. That is a lot of data to download on initial load, and unfair to those who don't check out past statistics.
Thirdly, even if were to allow for a large download, with the current structure choosing an arbitrary date on the homepage would re-run the parse, filter out the dates, re-calculate the statistics and supply it as updated props to the components. On selecting a date, there's a significant load in parsing this data repeatedly, which isn't helpful.
Hence, there was a need to re-evaluate our entire component and API structure to get the timeline to work.
Restructuring the API
One of the things we wanted to eliminate early on was to get rid of the parser in the frontend. After we receive a status 200 response, we should ideally be in a state to use the data immediately in our components. Parsing the data to fit the structure of our component on the frontend can be unproductive when we can have that data parsed and already structured in the first place when received from the backend.
We spent a few days studying each component and abstracting a standard data structure so that multiple components can immediately use it without going through another layer of parsing.
The next way through which we tackled redundancy was to remove calculable secondary data points from the payload. For example, active values can be calculated by using confirmed, recovered, and deceased statistics. Furthermore, in timeseries data, delta values can be calculated by subtracting adjacent cumulative values in O(1); however, it isn't the same the other way round since you'd have to loop an addition operation from 0 → n to reach your cumulative value every time.
While one can argue that we're introducing "parsing" again into the frontend, I guess that's the fine line we draw between using a little of the browser's computing resources and minimizing the download size of the JSON payloads.
But redundancy is not always a a bad thing. Sometimes we can take advantage of redundancy in data by using them as an excuse to build reusable components in the frontend. In the table component, there's no need for a state's row component to be any different than that of a district.
So component reusability (in our case, we've abstracted the cell) would look something like this below.
Coming back to implementing the timeline, we wanted to make sure that the data sent to the website carried a smaller payload in size with relevant data on initial load. Doing so meant that we had to move from relying on four endpoints to just one single endpoint.
At first, the idea seemed satisfying to arrive at just using one endpoint. However, we realized that we didn't have to fetch a considerable part of the response repeatedly when switching between the dates of the timeline.
The timeseries graphs and maps used this portion of data. One endpoint would've been the icing on the cake (with that said, it's also important to acknowledge that a single source of truth can also be a problem sometimes). But we figured that the timeseries data could reside on a separate endpoint that can be called once and don't have to react to changes in the timeline in terms of fetching new data henceforth.
So to address that, we had to split the final endpoint into two -
data.json. The former gets fetched only once, while the latter gets fetched every time we look back to a different date while using the timeline.
Finally, we partitioned
data.json by dates in the format of 'data-yyyy-mm-dd.json' to revisit the statistics of any past dates. So now, going back in time was just a matter of changing the endpoint's date.
Shoutout to @shuklaayush for working on the ultimate parser. Here's my v2 version of the parser at this link, but @shuklaayush beat me to it with his v3 parser by supporting districts and minimizing redundancy :P
Using the newly structured API
When the home component calls this endpoint, it receives the data and distributes it as props to the Table, Timeseries, and MapExplorer components. And if we make these components react to prop changes, then we're golden.
All we have to do is have the user specify a date, then update the endpoint URL and fetch.
When the new data is received, the home component performs a re-render and distributes a new set of props to the components. And since we've made the components to react to these prop changes, the components will have their states updated, and you now have the timeline implemented.
So the strategy in terms of code and the API has been figured out. Next comes the user interface.
At this point, I open Figma and have the mobile version of the website open in front of me. I immediately think of all the potential ways I could place the timeline.
I try to experiment with every possible way to pick the most straightforward design that would require the least explanation for someone to be able to use it.
After a couple of potential options, I ended up deciding to place it here.
That position also had the added benefit of users noticing the new feature immediately. (Left: Actions panel, Right: Timeline panel)
But all of this has just been me moving around rectangles and text boxes on Figma; I still have no clue about how I would end up turning this into code just yet.
(Maybe I had a little idea)
Bringing it alive
I figured out that I would have to switch to the timeline mode by clicking the history icon. So I created a state called
isTimelineMode to toggle between the Actions panel and the Timeline panel.
Next, I wanted the toggle to look a little bit more organic rather than just a component replacement. I figured a card flip animation would eventually do the trick. But I didn't know how to implement it at first, so I went around googling for examples for some inspiration.
And they sure did! I used one of their examples to understand how the library worked and how the flip animation worked and tweaked it to our requirement. While reading the documentation help in learning about extra configurations or get a deep understanding of what the library can do, learning through examples can help a lot.
After this, I needed to figure out a way to scroll through the dates. use-gesture is another incredibly well-made package that allows people to interact with components through gestures like drag, hover, scroll, wheel, etcetera. This library, coupled with use-spring, is a powerful combination for creating highly interactive components in React.
Funnily enough, I also used the help of another example on their website to learn how their API works and used it as a starting point to improvise upon it and emulate the functionality I wanted.
The timeline would behave like a carousel interface, except they are now dates instead of pictures you would see in a typical carousel.
After adding support for dragging on touch devices, I realized that using a pointer to drag while using a touchpad or mouse would be incredibly infuriating for desktop users. So the next step was to support controlling the timeline using the keyboard's LeftArrow to go left, RightArrow to go right, and Escape to reset to the current date. I used the useKeyPress hook available in react-use to add this functionality easily.
Adding a little more detail
By now, most of the work on implementing the timeline is done. At this point, I go over the code and try to abstract everything a bit more to avoid redundancy and profile for any performance bottlenecks to see if any extra re-renders are happening anywhere that I could potentially address and fix before the final merge.
Sometimes in the middle of doing all these things, I try to see if there are any possibilities of squeezing in another little enhancement somewhere.
Highlights, like the one shown above, was a result of that.
Push to production
After all of this, we create preview deployments and share it internally to test for any bugs that may have slipped during development. After a thorough review, we push to production and hope that nothing crashes🤞🏽