A friend took over a Developer Relations team and inherited the usual evidence of life: GitHub stars, Discord growth, registration counts, conference photographs and enough appreciative remarks to make every trip sound worthwhile. What he did not inherit was a way to tell which people reached by the programme went on to write code.
A $50,000 hackathon and a $5,000 meetup series made the problem difficult to ignore. The hackathon had a sponsor recap and photographs of a full room; the meetups had registrations and community growth. Either could furnish an agreeable paragraph for the quarterly review, but nothing in that evidence could rank them despite a tenfold difference in cost or say whether anybody shipped code afterwards.
More than fifteen years running Developer Relations made that collection immediately familiar. Rooms, registrations, stars and appreciative comments are useful operating signals, but I learned not to ask them to prove what happened next. I care about who built, who kept building and what the programme changed, because that is how I separate conferences worth revisiting from rooms which merely photographed well.
Turning my friend’s question into a measurement system required four definitions: which attendee could be resolved to a technical identity, what counted as subsequent work, how long that work persisted, and what proximity to an event could honestly establish. I built the dashboard to follow that chain from event records to public technical activity and later retention.
The problem extends well beyond one programme. The State of Developer Relations 2024 found that proving business impact had risen from seventh to second among the challenges practitioners named, at 28.5% of respondents; 61% said they found it difficult to prove their influence. Trust, education and relationships do not always produce joinable rows, but the observable part of the work can be made considerably less impressionistic.
Excluding employees changed the job
The dashboard has a toggle labelled “Exclude Employees”. The first time I used it, more than half of one ecosystem’s apparent activity disappeared. What had looked like a large external community was, in considerable part, a company with a Discord server and a very committed engineering team.
Employee work is real and may be exactly what an ecosystem needs, but removing it gave the DevRel team a different job. A community dominated by employees needs external adoption; one with many new but transient contributors needs retention; and one with established builders losing momentum needs attention before acquisition.
Aggregate growth had elided those distinctions; excluding employees changed both the size of the number and the intervention it called for.
Start with an action a developer took
Electric Capital’s 2024 Developer Report offered a firmer unit of measurement. It analysed 902 million public commits across 1.7 million repositories and mapped developers to the ecosystems where they worked. Its units are admirably ordinary: a monthly active developer made a public commit within a rolling 28-day window; a full-time developer recorded ten or more commit days in a month; tenure begins with the first observed contribution.
A public commit records a named person performing a technical action at a known time, a firmer starting point than a signup form, a Discord membership or the fact that somebody entered a conference room. Cause, usefulness and retention require other evidence.
The ugly work is identity resolution. Hackathon files arrive with arbitrary column names, several kinds of GitHub URL and email addresses that may not match a public profile. The resolver scans every column, extracts usernames even from repository links that no longer resolve, matches what it can against known developers, and consults repository contributor lists when the direct route fails. Each event can then show total attendees, attendees resolved to GitHub identities, new developers and the people still contributing afterwards.
Twenty-eight days after the event
For event analysis, the dashboard considers activity from fourteen days before an event to twenty-eight days after it. That window includes people who had already decided to build and misses the talk whose useful idea becomes a repository three months later. Comparisons between similar programmes become more persuasive when the difference persists through 28- and 60-day activity; an absolute activation percentage attached to one event deserves much less confidence.
Separate cohort views examine 30-, 60- and 90-day retention, which helps locate a different class of failure. If acquisition rises while retention falls, the programme may be filling a broken onboarding path more efficiently. If the next cohort persists longer, something in the product, documentation or support may have improved, although the dashboard alone cannot say which intervention deserves the credit.
Churn is a list of people
The churn score changed the work more than any graph. It combines recency, the direction of activity and changes in the size and frequency of commits, then returns a list of developers whose participation appears to be decaying.
“Community health” remains pleasantly abstract until it resolves into a developer with a substantial history whose work has slowed. That person already knows the platform, has endured its installation and error messages, and may be encountering a problem nobody has asked about; a thoughtful email asking what they are building can teach more than another generic message sent to ten thousand nominal members.
Some of the people in the dashboard contribute only to this ecosystem; others build across several. That changed the conversations I wanted the team to have. The first group may be ready for deeper opportunities, while the second are experienced developers still deciding where to invest their time. For Move developers, I can sometimes follow the trail beyond a commit to an on-chain package and its transactions. That data is incomplete, and the dashboard says so.
Retention changed the budget conversation
Once the hackathon and meetup attendees had been resolved to developers, the two line items could be compared by new builders, 28- and 60-day activity, and cost per retained developer. Cheapness alone did not decide the ranking; the programme with stronger retention earned the better position, which changed the budget conversation.
The evidence earned its keep by changing what the team did. The employee toggle revealed an acquisition problem, retention cohorts implicated onboarding, the churn list suggested particular conversations, and event comparisons identified which rooms deserved another visit. Not every useful relationship leaves a measurable trace, yet the ones that do no longer need to be defended with photographs and confidence alone.