An illustrated developer compares Cursor and GitHub Copilot alongside charts analyzing users by role and company attributes.
Every few weeks, a chart makes the rounds on developer social feeds claiming that Cursor is eating GitHub Copilot's lunch, or that GitHub Copilot is untouchable across the industry. Usually, the number comes from a public developer survey—and usually, it is answering a question nobody on your engineering team actually asked. A survey percentage tells you how many self-selected people who clicked a link answered a specific prompt in a specific month. It does not measure global market share, and if your Windows-heavy enterprise shop runs mostly C# and Java while the survey crowd leans toward solo JavaScript hackers, that headline percentage is practically fiction for your environment.

That composition gap sits underneath almost every Cursor versus GitHub Copilot debate. Take Stack Overflow's 2024 Developer Survey, which gathered 65,437 responses across 185 countries in May 2024: 74% of respondents reported using Visual Studio Code, reminding anyone paying attention that respondent pools skew heavily toward specific ecosystems before AI tools even enter the chat. Fast-forward to Stack Overflow's 2026 edition—fielded between June 23 and August 5, 2026, with 30,903 valid responses across 169 countries—and you see why context is everything. When TechRepublic covered the 2026 Stack Overflow results on November 20, 2026, it led with programming languages (JavaScript at 62%, SQL at 58.4%, HTML/CSS at 58.2%, and Python at 58%) rather than AI assistants. Meanwhile, Stack Overflow's own 2026 AI data tables show that among 12,255 respondents answering the coding-agents question, 65.5% used Claude Code, 58.7% used GitHub Copilot, and 25.5% used Cursor. Does that mean your 40-person enterprise dev team uses Claude Code more than twice as much as Cursor? Not unless your team mirrors that exact self-selected crowd.

To bridge the gap between a public survey sample and your actual engineering organization, you need raking—a classic statistical weighting technique also known as iterative proportional fitting. Building a clean, auditable raking pipeline takes roughly 200 lines of Python, 12 structured steps, and about 70 minutes of focused setup. Using a small Python package called survey_rake and a synthetic 4,000-developer dataset as a safe training ground, you can learn how to reshape survey respondents to match your organization's language, company size, and seniority mix—and just as importantly, learn when the math waves a red flag telling you not to trust the result.

An illustrated developer compares Cursor and GitHub Copilot alongside charts analyzing users by role and company attributes. Why Raw Survey Percentages Mislead Any Tool Comparison​

Before writing a single line of pandas code, you have to untangle what major developer surveys actually measure. In Stack Overflow's 2024 data, 62% of respondents (out of 60,907 answering the question) said they were currently using AI tools in development, up from 44% in 2023, while 76% said they were using or planning to use them. That 14-point gap separates current reality from future intent. In that same 2024 wave, 41% of ChatGPT users said they wanted to use GitHub Copilot the next year—a figure measuring desire inside one specific subgroup, yet precisely the sort of stat that gets mangled into a "41% market share" claim by careless commentary.

In 2026, Stack Overflow broadened its workplace AI prompt (n = 17,464), finding 65.9% using "AI coding assistants or coding agents," 62.5% using general-purpose AI chat tools, and 26.2% using AI agents or automated workflows, with 73.0% of coding-assistant users relying on them daily. JetBrains takes a different approach with its 2026 State of Developer Ecosystem report, marking ten years of annual surveys with 28,945 respondents after cleaning, and balancing data by geography, employment, programming languages, and JetBrains product usage. Over the decade, JetBrains expanded its questionnaire from roughly 150 questions to more than 500. More questions bring nuance, but they also ensure that two percentages from two different surveys rarely share the same wording, denominator, or recruitment bias.

Self-selection is the culprit that weighting cannot completely erase. People who fill out online developer surveys read developer blogs, follow community announcements, and care deeply about tooling. Reweighting cannot turn a self-selected poll into a true census or cure unmeasured selection quirks. What raking can do is stop a survey dominated by solo web developers from masquerading as an answer for an enterprise backend team. It requires only three short marginal target tables from your own organization rather than a full cross-tabulation, converges quickly on cooperative data, and produces one reusable weight per respondent. The price you pay is variance: rare respondents receive big weights, shrinking your effective sample size—a number most published hot takes conveniently leave out.

Prerequisites, Tested Stack, and Windows vs. Linux Notes​

The survey_rake tutorial runs on a lightweight Python data stack. While standard pandas and NumPy calls behave predictably across nearby releases, the tutorial outputs discussed here were produced on the exact Linux stack below:

ComponentVersion UsedRole in Pipeline
Python3.12.3Runs the survey_rake package and CLI
pandas3.1.1Loads CSV exports and parses multi-select columns
NumPy2.5.5Vectorized weights, bincount marginals, bootstrap resampling
pytest9.2.0Enforces five unit tests on raking math and error states
Operating systemLinux (POSIX shell)Reference environment for shell commands

If you are following along on Windows 11, you do not need to spin up a cloud VM. You can either run the exact POSIX commands inside Windows Subsystem for Linux (WSL), or run natively in Windows PowerShell using python -m venv.venv, activating with .venv\Scripts\Activate.ps1 instead of source.venv/bin/activate, and swapping head -4 for Get-Content data\synthetic_survey.csv -Head 4.

Expect to invest roughly 70 minutes end to end: 10 minutes for environment setup, 25 minutes for the data loader and raking engine, 25 minutes for diagnostics, bootstrapping, and team scenarios, and 10 minutes for output labeling and pitfalls review. You do not need to download a real survey file to learn the workflow, because Step 4 builds a synthetic 4,000-row dataset matching real multi-select CSV exports. If you later point the pipeline at real Stack Overflow data, note that Stack Overflow licenses its survey data under ODbL 1.0 and contents under DbCL 1.0, which carry attribution and share-alike obligations for derived datasets.

Steps 1–3: Define the Metric, Scaffold the Project, and Open a Ledger​

Step 1: Lock down one exact metric​

Before writing code, write one sentence defining what your pipeline estimates. Survey reports routinely mash together six distinct categories:

Metric CategoryQuestion ShapeWhat It SupportsExample from Public Surveys
AwarenessHave you heard of this tool?Brand reach, not adoptionN/A
Ever usedHave you ever tried this tool?Trial rate, not retentionRarely separated for coding assistants
Currently usesWhich tools do you use now?Active adoption among respondents62% used AI tools in SO 2024
Plans to useDo you use or plan to use?Combined adoption and intent76% used or planned in SO 2024
Wants next yearWhich tools do you want to use?Future desire inside a subgroup41% of ChatGPT users wanted Copilot in SO 2024
Paid seat shareWhich tier do you pay for?Revenue-side adoptionRequires vendor or billing logs

For survey_rake, lock in a single metric: the share of answered respondents who say they currently use each tool at work. That sentence becomes the mandatory basis label on every report.

Step 2: Create the project skeleton​

From your working directory, initialize a clean folder, virtual environment, and package layout. Using python -m keeps imports clean without requiring a formal pip install -e. step:

Code:
mkdir survey-rake-lab && cd survey-rake-lab
python3 -m venv.venv
source.venv/bin/activate
pip install pandas numpy pytest
mkdir -p survey_rake/tests data
touch survey_rake/__init__.py

Your completed repository will contain eight Python files and three data files:

  • data/ledger.json, data/synthetic_survey.csv, data/targets.json
  • survey_rake/__init__.py, cli.py, estimate.py, loader.py, rake.py, report.py, synth.py, and tests/test_rake.py

Step 3: Start a survey metadata ledger​

Create data/ledger.json before importing a single row. Record the publisher, edition, fieldwork window, sample count, license, exact metric wording, recruitment notes, and whether publisher weights already exist. Where you have not verified a field yet—such as JetBrains' 2026 license or exact question text—leave explicit "TODO" placeholders rather than guessing from memory. Keeping a fieldwork field separate from webpage update stamps also prevents a classic reporting blunder: a survey summary page refreshed in November 2026 may still sit atop data collected in May 2024.

Steps 4–6: Build Respondents, Normalize Multi-Selects, and Set Targets​

Step 4: Generate a synthetic respondent table​

Because raw market share is not the point of the exercise, survey_rake/synth.py builds 4,000 synthetic developers (using NumPy's default_rng(2026)) with three traits—language (Python, JavaScript, Java, Other), org_size (solo, small, mid, large), and experience (0-4, 5-14, 15+)—plus a semicolon-delimited ai_tools_work column.

Crucially, the generator plants realistic structural skew on purpose:

  • The sample itself over-represents JavaScript (35% target probability) and Python (30%), alongside solo (30%) and small (35%) shops, leaving Java (12%) and large organizations (15%) underrepresented.
  • Logistic probabilities (logit) nudge solo/small shops and junior (0-4) developers toward Cursor, while nudging large organizations and Java engineers toward GitHub Copilot.
  • If a respondent selects zero tools, the generator writes the literal string (none) instead of leaving a blank cell.

Run python -m survey_rake.synth data/synthetic_survey.csv to generate the test file. When moving to a real Stack Overflow or JetBrains CSV later, always inspect the survey's accompanying codebook first—column names change almost every year.

Step 5: Parse multi-select answers cleanly​

In survey_rake/loader.py, load_survey() reads the CSV, flags non-null entries in the tool column as answered, splits each semicolon-delimited cell, trims whitespace, and performs case-insensitive matching against an ALIASES dictionary ("Cursor": ["Cursor"], "GitHub Copilot": ["GitHub Copilot", "Copilot"]). By default (drop_unanswered=True), unanswered rows are dropped so every share uses answered respondents as its denominator.

Inspecting the unweighted synthetic baseline produces:

  • uses_cursor: 0.276 (27.6%)
  • uses_github_copilot: 0.335 (33.5%)
  • Language mix: JavaScript 36.0%, Python 29.4%, Other 22.6%, Java 12.0%
  • Organization size mix: small 34.3%, solo 29.8%, mid 20.2%, large 15.6%

At first glance, Copilot leads by nearly 6 percentage points. Hold that thought—because we haven't asked whose team we are measuring.

Step 6: Define team scenarios in data/targets.json​

Create data/targets.json with three target populations, each specifying marginal shares summing to 1.0 for language, org_size, and experience, along with a source note (such as "our HR export, Nov 2026"):

  1. Startup: 45% Python, 40% JavaScript, 5% Java, 10% Other; 10% solo, 70% small, 15% mid, 5% large; 40% junior (0-4), 45% mid (5-14), 15% senior (15+).
  2. Enterprise: 25% Python, 20% JavaScript, 40% Java, 15% Other; 2% solo, 5% small, 25% mid, 68% large; 20% junior, 50% mid, 30% senior.
  3. Balanced: 35% Python, 30% JavaScript, 15% Java, 20% Other; 20% solo, 30% small, 25% mid, 25% large; 30% junior, 45% mid, 25% senior (a control scenario near, though not identical to, the raw sample).

Never copy target distributions from the same survey you are reweighting—that circular trick merely launders sample bias. Keep three strict rules in mind: every category level present in the CSV needs a target entry, every positive target needs at least one matching respondent, and you should avoid exact 0.0 targets (notice Enterprise uses 2% for solo) because zero targets fight against lower weight-trimming bounds and prevent convergence.

Steps 7–8: Iterative Proportional Fitting and Convergence Tests​

Step 7: Build the raking loop​

In survey_rake/rake.py, _encode() validates target columns, checks that shares sum to 1.0 within 1e-6, and raises a RakeError on missing values, unmapped data categories, or empty target groups. The rake() function initializes every respondent at weight 1.0 and loops up to max_iter=200 times:

  • For each variable, np.bincount(c, weights=w, minlength=...) sums current weights per level and multiplies each respondent's weight by target_share * total_weight / level_weight.
  • After cycling through all three variables, it trims weights to a band around the mean (default trim=(0.2, 5.0)) so a handful of rare respondents cannot hijack the entire calculation.
  • It checks the worst remaining gap between weighted margins and target shares, stopping when gap < tol (1e-6).

Remember one structural boundary: raking aligns marginal distributions independently, not joint cells. If your enterprise team specifically employs mostly senior Java developers at huge banks, matching the Java margin and the large-company margin separately does not guarantee that specific intersection matches. If you possess a full joint HR table, combine them into one composite column (e.g., Java-large) and rake on that instead.

Step 8: Read diagnostics before percentages, and lock in pytest guards​

Running the startup scenario before examining tool percentages yields critical diagnostic signals:

  • The margin gap drops rapidly (0.0086 -> 0.0026 -> 0.0010 -> 0.0000) and converges cleanly.
  • Weighted org_size lands squarely on target (0.10 solo, 0.70 small, 0.15 mid, 0.05 large).
  • However, the weight distribution spreads from 0.20 up to 4.56, with a median weight of 0.48 sitting well below the 1.0 mean—a clear sign that most solo/other respondents are being downweighted while a smaller subset carries the heavy lifting.

Lock these invariants down in survey_rake/tests/test_rake.py with five pytest checks: weighted margins match targets, an overly tight trim ((0.5, 2.0)) honestly flags converged=False (leaving roughly a 7-percentage-point gap on this synthetic data rather than silently outputting wrong numbers), unknown levels raise RakeError, bad share sums raise RakeError, and equal weights yield an effective sample size equal to n.

Steps 9–10: Effective Sample Size and Re-Raked Bootstrap Intervals​

Step 9: Measure precision loss with Kish's effective sample size​

In survey_rake/estimate.py, weighted_share() computes (flag * w).sum() / w.sum(). Alongside it, kish_neff(w) computes the Kish effective sample size, w.sum() ** 2 / (w ** 2).sum(), which translates your weighted dataset into the equivalent number of unweighted respondents. Dividing the raw head count (4,000) by n_eff gives the design effect. A design effect of 2.0 means weighting just cost you half your statistical precision.

Step 10: Re-rake inside every bootstrap replicate​

Standard textbook confidence intervals (like an unweighted Wilson interval) assume equal weights and will falsely flatter your precision. Instead, bootstrap_ci() resamples n rows with replacement 500 times (using seed=7), re-runs rake() from scratch inside every replicate, records the weighted share, and pulls the 2.5th and 97.5th percentiles.

Re-raking inside each replicate is non-negotiable; resampling fixed weights ignores the variance introduced by estimating the weights themselves. The function also tracks failed_reps in case a random resample drops a rare level or fails to converge. One crucial statistical caveat for head-to-head comparisons: checking whether Cursor's and Copilot's individual 95% intervals overlap is an informal visual check, not a formal hypothesis test. If you want a rigorous test of the Cursor-versus-Copilot gap, compute (copilot_share - cursor_share) inside each bootstrap replicate and inspect the 95% interval of that paired difference.

Steps 11–12: Scenario Sensitivity, Late-2026 Product Churn, and Labeling​

Step 11: Run all three scenarios and read the swings​

Wire report.py and cli.py together so that any non-converged main rake raises a RakeError and forces CLI exit code 2 (handy for automated CI checks). Running python -m survey_rake.cli --csv data/synthetic_survey.csv --tool-col ai_tools_work --targets data/targets.json --scenario all reveals why one raw headline can never speak for every team:

Synthetic ScenarioTrim BandCursor Share (95% CI)Copilot Share (95% CI)Effective n (of 4,000)Design EffectMax WeightConverged?
Raw (unweighted)N/A27.6%33.5%4,000.01.001.00N/A
Startup(0.2, 5.0)30.1% (28.0–32.3%)31.1% (29.0–33.0%)1,880.62.134.56Yes (11 iters)
Balanced(0.2, 5.0)24.8% (23.5–26.2%)38.5% (36.9–40.0%)3,511.31.142.04Yes (3 iters)
Enterprise(0.2, 5.0)FailedFailed———No (gap 0.1365)
Enterprise (loosened)(0.05, 15.0)15.2% (12.9–17.8%)57.1% (53.6–60.6%)683.05.8615.00Yes (32 iters)

(Note: All figures above are synthetic outputs generated by the tutorial's planted associations, not independently executed real-world market shares.)

Look at what happened when the target team shifted. In the Startup scenario, Copilot's raw 6-point lead evaporates into a near dead heat (30.1% vs. 31.1% with heavily overlapping intervals). In the Balanced run, because the targets still pull slightly toward larger firms and Java compared to the raw sample, Copilot widens its lead to 38.5% vs. 24.8% with minimal precision loss (n_eff = 3,511.3). And in the Enterprise run, default trimming refuses to converge (gap = 0.1365 after 200 iterations) because the sample lacks enough large-company Java engineers. Loosening --trim-lo 0.05 --trim-hi 15 forces convergence, swinging Copilot to 57.1% and Cursor down to 15.2%—but at a brutal cost: effective sample size collapses to 683 respondents, design effect spikes to 5.86, and the largest weights slam into the 15.0 ceiling. A responsible analyst treats that Enterprise output as a directional hint, not a precise measurement.

Step 12: Enforce a five-item basis label amid fast product churn​

Never ship a weighted percentage naked. Attach five mandatory items to every figure: (1) survey and edition, (2) fieldwork dates, (3) exact metric sentence, (4) target scenario and HR/repo source, and (5) confidence interval plus n_eff and self-selection caveat.

That timestamp discipline matters more than ever in late 2026 because both products have mutated rapidly underneath the survey category of "AI coding assistant":

  • Cursor's November 2026 velocity: Following its Agent SDK, Cursor rolled out Build Live Apps on November 5, 2026—allowing cloud agents to scaffold, configure Cursor SSO authentication and Postgres databases, run security checks, and deploy live web apps—and launched two-way Voice Chat with Agents on November 6, 2026 across the Agents Window, IDE, web, and iOS. (While unconfirmed social rumors circulated on November 20–21 about a possible Cursor rename, always track official product names across survey waves.)
  • GitHub Copilot's November 2026 releases: GitHub introduced Copilot as a default agent in JetBrains Air via the Agent Client Protocol on November 3, brought Copilot into Xcode's native Coding Intelligence assistant in public preview (Xcode 27.3 beta, macOS Tahoe 26.6+) on November 4, retired older models on November 16, and rolled out GPT-6.2 Astra and GPT-6.2 Sol to general availability on November 17 (followed by Gemini 4 Neon on November 18).
  • Billing tier blind spots: Since June 1, 2026, GitHub meters Copilot chat, agents, and reviews in AI Credits worth $0.01 each across Free ($0, 2,000 completions/month), Pro ($10/month with $15 in total credits), Pro+ ($39/month with $70 in total credits), Max ($100/month with $200 in total credits), Business ($19/seat/month), and Enterprise ($39/seat/month). Yet a respondent on Copilot Free and a power user burning 20,000 credits on Copilot Max both count identically as "one user" in a binary usage survey.

Nine Pitfalls, Ten Troubleshooting Fixes, and Next Steps​

When numbers look fishy, audit your pipeline across three failure zones:

Data, Weighting, and Reporting Pitfalls

  • Data traps: Never confuse blank cells (NaN) with explicit (none) answers, which silently inflates your denominator. Double-check whether a published survey percentage divides by all respondents or answered respondents before crying foul. And strip "Copilot" from ALIASES if a real questionnaire lists Microsoft 365 Copilot or Copilot Studio alongside GitHub Copilot.
  • Weighting traps: Never silence convergence flags; over-tight trimming injects silent bias. Avoid zero targets and sparse levels, and cap raking variables at three if design effect climbs past 3 or 4.
  • Reporting traps: Never call a reweighted survey share "market share." Never plot different survey waves (e.g., May 2024 Stack Overflow and 2026 JetBrains) on one axis without prominent footnotes explaining fieldwork, recruitment, and wording differences. And never call overlapping startup intervals a decisive win without testing the paired bootstrap difference.

Quick Troubleshooting Map

  1. RAKE FAILED (max_gap > 0.01): Loosen --trim-lo/--trim-hi, merge sparse levels, or admit your target team differs too drastically from the sample.
  2. RakeError: data has levels with no target: Normalize casing/whitespace (df[col].str.strip()) and check for mismatches like Javascript vs. JavaScript.
  3. RakeError: target levels with no respondents: Merge zero-count target categories into Other.
  4. RakeError: shares sum to something other than 1: Set the final category as 1 - sum(others) to kill rounding drift.
  5. KeyError: tool column not in columns: Print list(df.columns) and verify exact spelling in the survey codebook.
  6. Every tool flag is False: Inspect a raw multi-select cell and pass the true delimiter to load_survey(..., sep=",").
  7. ModuleNotFoundError: survey_rake: Confirm survey_rake/__init__.py exists and run commands from the project root using python -m.
  8. RuntimeError: no converged replicates: Your trim band or sparse levels are failing across all bootstrap draws; fix the baseline rake first.
  9. Slow bootstrap: Prototype with --reps 100 on large real CSVs before running 500+ replicates for final figures.
  10. Wild weighted swings: Check n_eff and w_max immediately—if n_eff drops to a few hundred and weights hit the ceiling, scale back the target gap.

Once your tests pass on synthetic data, you can point the exact same CLI at real anonymized Stack Overflow or JetBrains exports, save per-respondent weights with pinned random seeds for reproducibility, suppress subgroup readouts where subgroup n_eff falls below your threshold, and cross-check weighted survey estimates against your organization's actual IDE seat billing and telemetry logs. Survey reweighting will never replace real seat audits or hands-on coding benchmarks—but it will stop a skewed internet poll from dictating your next developer tooling decision.

 

References

  1. Cursor vs Copilot Survey Reweighting: 12 Steps, 70 Min shattered.io 2026-11-22T01:41:49+00:00
  2. github.com docs.github.com
  3. github.com github.com