Polis v10 (that's binary I suppose)

[colinmegill]

“What’s next for polis?” An attempt at a casual, but dense, answer here in issues, as we approach a polis-1.0.0-stable after 11 years of development, and after an exciting year of technological advancements related to processing text.

At a high level, from my perspective, the next complete versions of machine learning augmented deliberation systems will have the following principles and aspects:

1. Reduce reporting costs by 10,000x to 1,000,000x

• A high quality process involving a polis conversation costs on the order of $100k to run — since it’s open source and people will participate freely if they feel it’s meaningful, the costs are technical personnel and time. Technical users still need to spend lots of time writing the results up into a report for consumers of a conversation.
• Ultimately, a solution will probably cost somewhere between ten cents and ten dollars based on how much is done and how efficient algos get (I’m assuming an LLM will be a part of the solution, will be run after every vote but be burned onto a chip, three years from now).
• See: Automatic text summarization of results #915
• See: https://demos.co.uk/wp-content/uploads/2020/12/Polis-the-Political-Process-NEW.pdf
• See: https://twitter.com/AnthropicAI/status/1671950647393124352

2. Integrate additional signals

• A participant may agree or disagree with a statement, but is it important to them? Tradeoffs count as additional signals.
• See: Allow participants to prioritize statements  #217
• See: https://en.wikipedia.org/wiki/Wiki_survey

3. Conquer “initialization”

• Current polis conversations need to be ‘set up’. Exercises seen as politically legitimate are usually ‘initialize → a process → conclude’.
• Path 1: new systems are “always on” or can be initiated by participants, think subreddits, — see: “‘Coherent Mode’ for the World’s Public Square” https://arxiv.org/abs/2211.12571
• Path 2: discrete conversations still exist, but initialization is more automatic, with time intensive steps like seeding existing perspectives handled via automatic processes, — See: https://papers.societylibrary.org/papers/diablo_canyon/map (click to expand all arguments mapped from all sources)

4. Use people’s time more efficiently

• Funnily enough, polis usually collects more votes than it needs! People get really excited to participate when the exercise is legitimate, transparent, and connected to real outcomes.
• This means future systems will have people do other tasks (like rating summaries, discussing tradeoffs, being involved with moderation, or participating in real and simulated small group discussions) and predict most votes.
• See: https://ai.facebook.com/research/cicero/
• As systems move to predicting votes, they’ll stay safe by giving participants more control over how they are represented, by allowing them to monitor the outputs which represent them. Self-rated representation can be done recursively at varying group sizes and varying fidelity of summary.

5. Identify experts and smaller groups

• Polis doesn’t always have metadata to rely on as a filter, and it's safe to assume worldviews will cross metadata sometimes, so, unsupervised approaches here are still interesting.
• Example: Let’s say, as an example, groups of teachers participating in a conversation about education have a very specific and different world view and vocabulary and specific knowledge… but there are only 7 shop/trade education teachers in a conversation of 1000 people. It’s a research problem to minimize noise (we don’t want 1000 / 7 groups) and maximize interpretability (200 groups isn’t going to be useful to interpret the results, which is why k-means is locked to 2-5 clusters at present).
• Perhaps this will even be aware of similarities in the semantic aspects of the feature space. That is, topic aware clustering might take into account whether the statements are saying similar things.

6. Plumb more of the space of opinion

• Active, and agentic elicitation “interesting, tell me more!” and “what about…” by the system itself is a wide ranging surface area to extend the number of comments submitted and the topics and possible solutions explored.
• See: https://openai.com/blog/democratic-inputs-to-ai for an example of agentic elicitation
• See: Agentic Elicitation #1727