Pilot · Mid-sized city — Planning department
Source codeTheme analysis of public comment
Result in one line: 4,100 comments on a draft zoning update summarized in two days instead of six weeks
An open-source notebook that groups the comments received on a plan or rule into themes, counts them, and links every theme back to the comments it came from so a reader can check it.
Published July 21, 2026 Updated August 12, 2026
- Skills needed to set it up
- Analyst or data scientist
- Readiness
- Guided setup Needs customization Human review built in
- Data it touches
- Public data only
- Who sees the output
- Public-facing
- License
- Other open license
- Cost to stand up
- No new spend
- Portable to other platforms
- Yes — platform-agnostic
- Use case category
- Coding & brainstorming
- Reviews it went through
- Community or advisory review Equity impact assessment Records retention review
- Security review
- Not reviewed
What do these mean?
- Analyst or data scientist
- Someone who works in Python, R or SQL.
- Guided setup
- A wizard or script walks you through installation.
- Needs customization
- Your team will need to adapt it before use.
- Human review built in
- A person checks the AI's output before it is used.
- Public data only
- Only data that is already public.
- Public-facing
- Residents or the general public interact with it.
- Other open license
- Name it in the access terms below.
- No new spend
- Built with licences, staff and infrastructure the organization already had.
- Yes — platform-agnostic
- Runs on any comparable stack with configuration changes only.
- Coding & brainstorming
- Writing or reviewing code, analysis, and idea generation.
- Records retention review
- Confirmed how long the outputs must be kept and where.
- Not reviewed
- Nobody has looked at this project on the catalog's behalf. Treat the link as a starting point for your own review, not as a recommendation.
Problem
A draft zoning update drew 4,100 comments across a web form, email and three open houses. Two planners read all of them and wrote a summary, which took about six weeks and arrived after the council had already held its first discussion.
The deeper problem was that nobody, including the planners, could check the summary. It said things like “many commenters raised concerns about parking.” Many is not a number, and there was no way to get from that sentence back to the comments it came from.
What we built
A notebook that produces a theme report. It embeds every comment, clusters them, proposes a label for each cluster, and writes out a table: theme, number of comments, number of distinct submitters, number of form letters, a representative quotation, and a link to the full list of comments in that theme.
A planner reviews and edits every theme label before publication, merges clusters that are the same argument, and splits ones that are not. That review is where the judgement lives, and it takes about a day.
How it works
Embeddings and clustering run locally on a laptop with open-source models — no comment text needs to leave the building for the part that produces the counts. The only step that calls a hosted model is the first-draft theme label, and that step is optional; the labels can be written by hand and the report is identical in every other respect.
The design rule we kept coming back to: every number in the report must be clickable. If the report says 312 comments raised setbacks, a reader can open all 312. That constraint killed several fancier ideas and is the reason we would defend the tool in a hearing.
Results
The zoning comment set was summarized in two days instead of six weeks, and the report went to the council with the staff recommendation instead of after it.
The published themes surfaced two arguments the manual summary from a comparable earlier process had missed entirely, both of them from a small number of comments — six and eleven — that got lost in the volume the first time round. One of them changed a provision in the final draft.
Clustering is genuinely bad at short comments. Anything under about fifteen words lands in a low-confidence bucket that a planner reads individually; that bucket was 9% of the comment set and there is no clever fix for it in the tool.
Lessons learned
Publish the method note alongside the report. Ours states what the tool does, what it cannot do, and that a human edited every theme label. The first question at the public meeting was “did a computer decide what we said,” and the answer needed to already be written down.
Never report themes without also reporting distinct submitters and form letters. A single number is a summary someone can game, and we would rather hand a reader three honest numbers than one clean one.
Keep the counting step local and deterministic. It made the privacy conversation trivial and it means the report can be regenerated by anyone with the comment file, which is the definition of checkable.
How to reuse
The repository has a worked example over a published comment set, so you can run it end to end before pointing it at your own. We took the transcript-splitting approach and the “every claim links to its source” rule from the clerk’s meeting summary entry in this catalog; that rule is the one thing we would insist on carrying over. An analyst comfortable with Python has this running in an afternoon.
About
- Area of work
- Policy & planning
- Data & informatics
- Communications & outreach
- Review status
- Reviewed & approved
How it's built
- How AI is involved
- Both
- Types of AI
- Classification & NLP
- Generative text (LLM)
- AI tools & models
- Python
- sentence-transformers
- Jupyter
- Claude
- Where it runs
- Desktop or local
Sharing & licensing
- Access terms
- Released under the European Union Public Licence 1.2, which is compatible with reuse by other public bodies and is what our legal office was comfortable with. The repository is public; no request needed.
- Portability notes
- A notebook and a small Python package. Embeddings run locally on a laptop; the summarisation step is one function you can point at any model provider, or replace with a human writing the theme labels by hand — the counts do not depend on it.
- Adapted from
What it took
- Cost to keep running
- No ongoing cost
- How it was bought
- No procurement needed
- Who it affects
- Public comment is one of the places where volume quietly beats substance: an organized campaign submitting 900 identical letters used to dominate a staff summary simply by being unmissable. Counting themes rather than letters cuts both ways, so the report always shows theme counts, distinct-submitter counts and form-letter counts side by side, and never collapses them into one number. A comment that appears once is still in the report, with its text, because in a land use process the single most useful comment is often the one nobody else made.
Data & access
- No PII/PHI in the shared material
- Yes
- Data sources
- Comments submitted to a published consultation
- Comment intake form metadata (channel, date)
- Data-governance caveats
- Comments submitted to a formal consultation are public records, and the theme report is published as part of the record. Submitter names and addresses are stripped before any text reaches a model and are never shown in the report, even though they are technically public — the report is a summary of what was said, not a directory of who said it. Comments are retained under the consultation's own schedule; the tool keeps no copy.