Executive Summary

When Plandek decided to completely rebuild their core platform in React.js, they needed to transform their team from junior to senior React developers as quickly as possible.
Features are now built 4x faster
Codebase quality improved and tech debt reduced
Delivery time reduced by more than 75%
Significant financial savings

Background

In late 2019, Plandek decided to switch from Ember.js to React.js. Rebuilding a core platform on a new tech stack is a significant decision, and the drivers of this change included:
Customer needs
They wanted to quickly adapt to changing customer needs as they grew
Hiring
They were having trouble finding Ember developers
Velocity
Feature delivery was slow, painful and expensive
Plandek provide businesses with end-to-end metrics and analytics. So, we thought they'd be a pretty great company for a case study.

Challenge:

Feature delivery was slow in part because the tech stack was inflexible, which makes it less appealing for devs to work with; hiring was slow in part because fewer devs want to work with Ember.

All of this impacted Plandek's ability to swiftly adapt to changing customer needs.
When Plandek made the call to rebuild their platform in React.js, they faced another critical decision. Should they:
A) Hire a React contractor for 6 months or more (£60-80k) and hope that by the end their team will have learned React by osmosis;
B) Train their current team in React so they can make the switch.

Reuben’s choice:

Head of Engineering, Reuben Sutton, chose to train the team using Skiller Whale, opting for 1-hour sessions every few weeks over 8 months.
There's never a time I could reasonably block out a team for 3 days for classroom training. An hour once a fortnight is easy.

Solution:

Skiller Whale was chosen because it provides flexible live training for dev teams led by a language expert. By conducting a detailed assessment, Skiller Whale gets inside the brain of each developer, so they only attend the sessions they would benefit from.
The sessions are hands-on with an expert, so Plandek's devs learned by doing, writing code in their own environment.
1
Sessions are selected based on assessing each individual on the team.
2
Challenging code exercises are designed to stretch learners.
3
Every session is live, with an expert.

Metrics for Impact

For the purposes of this case study, impact and gains from training were measured using the following data:

Quantitative

1. 'Before' and 'After' metrics for the team's output (cycle time; delivery time).
2. 'Before' and 'After’ skills and knowledge assessment in React by Skiller Whale

Qualitative

1. Evaluation by the tech manager, reflecting on the team's coding competence before and after training
2. Self-evaluation of learners on the impact of training on their skills

After Evaluations

The 'After' evaluations were done 3 months after training was completed in order to get a clear picture of the extent to which training has had lasting effects on the skills and competencies of the learners.
By looking at each of these measures of success, we aimed to build up a full picture of the impact of training on the team, as well as the company.
We found that the team's skills as assessed by their manager, themselves and the Skiller Whale React assessment, showed that there had been significant improvement as a result of training.
By doing a 'before' assessment, Skiller Whale were able to focus on the areas that the team needed help with the most. Reuben commented that:
The training plan they produced was exactly what my team needed

Skiller Whale React Assessment before training

We asked:
The Skiller Whale assessment asks practical questions about different features of the language, and also measures the learner's confidence in various techniques.
We found:
While the Plandek team started out relatively strong on React fundamentals, there was significant room for improvement across a range of other aspects of the language.

Scores in react.js assessment

Before training
After training

75%

90%

React Fundamentals

62%

84%

Advanced React

32%

82%

Code Perfomance

35%

85%

Debugging

50%

75%

Testing

56%

86%

Tech Debt Avoidance

Manager Assessment of Team Skills

The Head of Engineering, Reuben, assessed the team's prior React skills to be 'mostly junior' before training and 'mostly senior/expert' after training.
He assessed their general coding competency (not just in React) to be moderate (3/5) before training and good (4/5) after training.

Learner Self-Assessment of Skills

The Plandek developers on average assessed themselves as moderate (2.5/5) in React prior to training, and good (4/5) in React after training.
All learners either agreed or strongly agreed that the entire development team's coding skills have improved as a result of Skiller Whale training.
All learners noted that the overall quality of production code has improved as a result of Skiller Whale training.

Results #2

User Stories and bug fixes delivered much faster

Comparing the period before Skiller Whale training (Oct'19-Jan'20), the delivery time (measured in days on the Y axis) for features, and also bug fixes, has decreased significantly.
Head of Engineering, Reuben, noted that he had actually expected a big increase in Delivery time from February - July as the team began to adopt React.js.
Instead, he saw it steadily decrease as the team picked up React.js much quicker than he had expected. He also attributes the overall trend in decreased delivery time to the fact that Skiller Whale training improved the overall coding chops of the Plandek dev team.
During the training period (Jan-Aug), we can see a trend towards shorter cycle times, so that if we compare average days from one year to the next, we see a reduction from over 45 days to just over 10 days.
Cycle time tracks the time from when work starts on a unit of work to when the code is deployed. Comparing the period before Skiller Whale training (Oct'19-Jan'20), cycle time was higher than Plandek wanted it to be.
Reuben also noted that the units of work at this time were relatively simple (expected to take 1-2 days), and were taking a lot longer. He illustrated this with an example:
Last spring, we tried to add self-configuration into the platform. After 10 weeks, we were nowhere and I killed the project. After Skiller Whale training, the team did an equivalent project in 3 weeks - and it worked.

Indirect Business Impact

Unlocked a huge talent pool and were able to hire in 1 week
Significant financial savings
As a result of a successful transition to React, Plandek has seen a huge positive impact on customers, and has been able to hire great people in less than a week, where previously they were trying (unsuccessfully) to hire for months. In evaluating the decision to use Skiller Whale, Head of Engineering, Reuben said:
Skiller Whale has saved us many times what we’ve spent on them.

Conclusions

Through working with Skiller Whale, Plandek were able to not only rebuild their core platform in a new technology, but were able to level up their development team and quickly reap the rewards of that in code quality and velocity.