Conference Roulette: Random Papers and Attendee Votes
This may be the simple and best suggestion so far. A slight variation to avoid gaming:
- Authors simply submit arxiv link. There are no reviewers.
- Each attendee gets to see K random papers in their chosen areas and casts votes.
- Orals are granted to N papers by most votes. https://x.com/jeffbigham/status/1234296538911903745
Q. How do authors get feedback and fix errors/omissions? A. Arxiv should provide a link to some external discussion tool for every paper. Discussions for a paper should not be tied to a single conference and should be ongoing.
Q. What are the incentives for attendees to vote? A. Registration discount, special badge, a special event with authors, right to vote for the award. Each submitter should be required to vote (i.e. the price of submission).
Q. How does someone measure the value of paper if they don’t know where it was “published”? A. Citations, PageRank, awards, number of votes.
Q. Can this really work? A. NeurIPS had 13K attendees, 6.7K submissions. If only 10% attendees chose to vote, it is just 5 papers/attendee
Q. But wouldn’t people than just write sensational but bad papers just to get the popular vote? A. Once a paper gets selected for oral, one should expect more people to read it and use Arxiv linked discussions. Again citations should be the metric, not orals in conf X.
Q. What then really a conference is? What remains? A. Just a social gathering, opportunity to meet with authors, ask questions, potentially collaborate and catching up in general. Conferences should seize to become highly resource-constrained noisy gatekeepers of content.