There's never a time I could reasonably block out a team for 3 days for classroom training. An hour of focused coaching once a fortnight is easy.
Executive Summary
When Plandek decided to completely rebuild their core platform in React.js, they needed to transform their team from junior to senior React developers as quickly as possible.
Features are now built 4x faster
Codebase quality improved and tech debt reduced
Delivery time reduced by more than 75%
Significant financial savings
Background
In late 2019, Plandek decided to switch from Ember.js to React.js. Rebuilding a core platform on a new tech stack is a significant decision, and the drivers of this change included:
Customer needs
They wanted to quickly adapt to changing customer needs as they grew
They wanted to quickly adapt to changing customer needs as they grew
Hiring
They were having trouble finding Ember developers
They were having trouble finding Ember developers
Velocity
Feature delivery was slow, painful and expensive
Feature delivery was slow, painful and expensive
Plandek provide businesses with end-to-end metrics and analytics. So, we thought they'd be a pretty great company for a case study.
Challenge:
Feature delivery was slow in part because the tech stack was inflexible, which makes it less appealing for devs to work with; hiring was slow in part because fewer devs want to work with Ember.
All of this impacted Plandek's ability to swiftly adapt to changing customer needs.
All of this impacted Plandek's ability to swiftly adapt to changing customer needs.
When Plandek made the call to rebuild their platform in React.js, they faced another critical decision. Should they:
A) Hire a React contractor for 6 months or more (£60-80k) and hope that by the end their team will have learned React by osmosis;
B) Upskill their current team in React so they can make the switch.
Reuben’s choice:
Head of Engineering, Reuben Sutton, chose to upskill the team using Skiller Whale, opting for 1-hour coaching sessions every few weeks over 8 months.
Solution:
Skiller Whale was chosen because it provides flexible live team coaching for dev teams, led by a language expert. By conducting a detailed assessment, Skiller Whale gets inside the brain of each developer, so they only attend the sessions they would benefit from.
The sessions are hands-on with an expert, so Plandek's devs learned by doing, writing code in their own environment.
1
Sessions are selected based on assessing each individual on the team.
2
Challenging code exercises are designed to stretch learners.
3
Every session is live, with an expert.
Metrics for Impact
For the purposes of this case study, impact and gains from coaching were measured using the following data:
Quantitative
1. 'Before' and 'After' metrics for the team's output (cycle time; delivery time).
2. 'Before' and 'After’ skills and knowledge assessment in React by Skiller Whale
Qualitative
1. Evaluation by the tech manager, reflecting on the team's coding competence before and after training
2. Self-evaluation of learners on the impact of training on their skills
After Evaluations
The 'After' evaluations were done 3 months after coaching was completed in order to get a clear picture of the extent to which training has had lasting effects on the skills and competencies of the learners.
By looking at each of these measures of success, we aimed to build up a full picture of the impact of training on the team, as well as the company.
We found that the team's skills as assessed by their manager, themselves and the Skiller Whale React assessment, showed that there had been significant improvement as a result of live team coaching.
By doing a 'before' assessment, Skiller Whale were able to focus on the areas that the team needed help with the most. Reuben commented that:
The coaching plan they produced was exactly what my team needed
Skiller Whale React Assessment before training
We asked:
The Skiller Whale assessment asks practical questions about different features of the language, and also measures the learner's confidence in various techniques.
We found:
While the Plandek team started out relatively strong on React fundamentals, there was significant room for improvement across a range of other aspects of the language.
Scores in react.js assessment
Before training
After training
75%
90%
React Fundamentals
62%
84%
Advanced React
32%
82%
Code Perfomance
35%
85%
Debugging
50%
75%
Testing
56%
86%
Tech Debt Avoidance
Manager Assessment of Team Skills
The Head of Engineering, Reuben, assessed the team's prior React skills to be 'mostly junior' before coaching and 'mostly senior/expert' after Skiller Whale's live team coaching.
He assessed their general coding competency (not just in React) to be moderate (3/5) before coaching and good (4/5) after coaching.
Learner Self-Assessment of Skills
The Plandek developers on average assessed themselves as moderate (2.5/5) in React prior to coaching, and good (4/5) in React after live team coaching.
All learners either agreed or strongly agreed that the entire development team's coding skills have improved as a result of Skiller Whale coaching.
All learners noted that the overall quality of production code has improved as a result of Skiller Whale coaching.
Results #2
User Stories and bug fixes delivered much faster
Comparing the period before Skiller Whale training (Oct'19-Jan'20), the delivery time (measured in days on the Y axis) for features, and also bug fixes, has decreased significantly.
Head of Engineering, Reuben, noted that he had actually expected a big increase in Delivery time from February - July as the team began to adopt React.js.
Instead, he saw it steadily decrease as the team picked up React.js much quicker than he had expected. He also attributes the overall trend in decreased delivery time to the fact that Skiller Whale coaching improved the overall coding chops of the Plandek dev team.
During the training period (Jan-Aug), we can see a trend towards shorter cycle times, so that if we compare average days from one year to the next, we see a reduction from over 45 days to just over 10 days.
Cycle time tracks the time from when work starts on a unit of work to when the code is deployed. Comparing the period before Skiller Whale training (Oct'19-Jan'20), cycle time was higher than Plandek wanted it to be.
Reuben also noted that the units of work at this time were relatively simple (expected to take 1-2 days), and were taking a lot longer. He illustrated this with an example:
Last spring, we tried to add self-configuration into the platform. After 10 weeks, we were nowhere and I killed the project. After Skiller Whale coaching, the team did an equivalent project in 3 weeks - and it worked.
Indirect Business Impact
Unlocked a huge talent pool and were able to hire in 1 week
Significant financial savings
As a result of a successful transition to React, Plandek has seen a huge positive impact on customers, and has been able to hire great people in less than a week, where previously they were trying (unsuccessfully) to hire for months. In evaluating the decision to use Skiller Whale, Head of Engineering, Reuben said:
Skiller Whale has saved us many times what we’ve spent on them.
Conclusions
Through working with Skiller Whale, Plandek were able to not only rebuild their core platform in a new technology, but were able to level up their development team and quickly reap the rewards of that in code quality and velocity.