- Q: What's the point of this?
- A: The idea is directly inspired by the "Superforecasters" – people who try to rigorously get less worse at predicting future events. SteamProphet is an adaption of their methods for gaining business intelligence about games on Steam. Not only is it a useful way to gut check your instincts (I found mine to be laughably bad), but it's also a great way to keep your finger on the pulse of what's actually happening in the market, and to generate educational discussions with your similarly-minded friends and colleauges.
- Q: Can I join your SteamProphet group, Lars?
- A: Yes... assuming I have room. For now just enter your contact information on this form and I'll get back to you if I have extra capacity in one of my groups.
- Q: Can I start my own group?
A: Yes! We will eventually provide a full set of tools/resources on this page to help, but until then you'll have to run it by hand, same as I've been doing.
To get you started, here's some helpful tips on running a SteamProphet group.
- Q: I have a bunch of friends who want to start a group. Will you run it for us?
- A: Maybe!...eventually. Once I have the right automated tools in place I'll see what my capacity is. SteamProphet groups generate a lot of good discussion and I'm happy to help facilitate that; I just don't want to spread myself too thin. Send me your information in the form linked on the top question and we'll see how it goes.
- Q: Will you ever create a web version of SteamProphet?
- A: Eventually this site will have a login system. We will still run all groups privately, though.
- Q: Did games that score 0 SteamProphet points make 0 money?
- A: We do not -- and cannot -- know for sure, and I cannot emphasize this enough. The SteamProphet score is an aggressively conservative estimate -- a lower bound, or "floor" if you will, on a game's earnings. It means "we're pretty sure the game made at least this much." Some developers whose games scored 0 SteamProphet points have shared their real stats with me, and in practice this tier seems to make somewhere between nothing at all and a few thousand dollars, but they could always have made more. Our score intentionally and aggressively underestimates actual results.
- Q: Did games that score 10,000 SteamProphet points make 10,000 dollars?
- A: We're pretty sure they grossed at least that much, yes (see above point). However, we will need to run some empirical studies on our guesses to know exactly how sure we are. So if you have a game that launched since we started tracking, and you want to share your sales results at the four-weeks-since-launch mark, we'd really appreciate it. Email me at lars dot doucet at gmail dot com.
- Q: Does SteamProphet tell me how well a game developer has done overall?
- A: No. Super No. We only track scores at the one month mark past release. Steam has been optimizing their algorithm to push long-term sales these days, and even my game made the vast majority of its revenue long after initial release. Also, games often release on other platforms (PSN, Xbox Live, Nintendo eShop, GOG, etc) that we don't track at all. And keep in mind that Steam takes a cut of the gross sales, and some games have publishers that take a cut, etc. Also we have no idea how much a developer spent to develop their game, so we have no way of knowing whether they turned a profit or anything like that -- a tiny game grossing $1K could have made a profit and a big-budget $1M-grossing game could still be $10M in the red. SteamProphet scores really only tell you how well certain games have done in comparison to one another, on Steam, in their first month, in terms of minimum estimated gross revenue. Also, "doing well" is a subjective measure. It would be really great if we had a way to easily measure "met/exceeded the developer's expectations," but that's not easy to quantify or to track. Interpretation is key.
- Q: Do you have plans of turning SteamProphet into a for-profit enterprise?
- A: I have no such plans. The most I can foresee is starting a Patreon to cover web development costs.
- Q: You run your group once a week – isn't that inaccurate? Doesn't it favor games released on Monday?
- A: That's correct! Running a group once a week is a compromise with convenience to get people to actually participate (once a week is a much easier commitment than once a day). It's also far easier for me to calculate all the scores four Sundays later than to manually keep track of each one and score it exactly 28 days after its actual launch date. This means that games released on Monday have a few extra days to rack up a higher score; in practice this doesn't really make much difference. First, most game releases are clustered towards the beginning of the week, and almost nothing releases on the weekend, so the advantage isn't huge. Second, it's almost never a close call between two games -- the scores diverge so wildly that a few extra days usually isn't going to make a difference. If/when I start running groups with automated computer assistance, I can eliminate this scoring glitch, but for now it's a compromise I'm willing to make.
- Q: Are you encouraging gambling based on the success of indie games?
- A: Absolutely 100% not. I do not tolerate gambling or betting in any of my groups. There's multiple reasons for this -- first, I have a multitude of personal and ethical concerns with gambling in general, and I'm also concerned with the toxic (and even potentially criminal, in the worst case) effects that sort of thing can have on a community. I also don't want to encourage new and creative vectors for fraudulent low-margin games on Steam. Also, gambling makes things really competitive, which makes people less likely to engage in open and friendly discussion, which I have found to be one of the most valuable things about this whole project. I obviously can't control what people do with things they organize themselves, but gambling is forbidden in all of my groups, and I strongly warn others against it for the above reasons.
- Q: Is this about indie elites making sport of other developers' livelihoods?
A: No. My vision is the opposite -- it's about empowering every developer to sharpen their business skills and share information about the giant, terrifying marketplace that holds all our fates in its hands like tiny fragile birds. It's also about giving smaller developers a chance to find a community of like-minded people they wouldn't ordinarly interact with, and learn from each other via discussion. We often spend years developing our games, but many developers -- especially new ones -- don't even know where to start when it comes to learning business intelligence skills, or making connections with other developers. And the market is brutal. I want to help people learn the lay of the land.
That's also why the scoring algorithm is very intentionally a "hit detector" rather than a "miss detector" -- we estimate aggressively pessimistic lower bounds, the least a game has probably made, when in reality it has almost certainly made more. Low-scoring games are the ones we know the least about. We could have created a "miss" detector by calculating the opposite -- an aggressively optimistic upper bound -- where a low upper bound would signify a possible flop, but for many, many, reasons, we have chosen not to do this. Not the least of which is the much larger amount of uncertainty involved in that particular calculation.
That said, I got my start in educational and serious game design, so I flatly reject any notion that "playing games" with serious topics is fundamentally disrespectful, while acknowledging that this perception is nonetheless fairly automatic and widespread and something we must be sensitive to, because communication is important.
Tone, optics, and culture are as important as intentions, and require vigilance and care. I will not tolerate mocking, jeering, or derision in any of my groups, and I encourage the same in public discussions about SteamProphet. This is about learning and lifting up, not about hoarding information or putting others down. We cannot escape analyzing and criticizing individual games, but anyone who doesn't have basic compassion for their fellow developers is unwelcome in my groups.
- Q: Isn't it uncouth to play this game publicly?
A: I'm coming around to this position in the wake of the project's initial release. I was initially tweeting my list of SteamProphet picks, but now I'm thinking I'll only post actual pick lists to my private groups, though I will continue to discuss my thoughts on upcoming games publicly (which is what I've been doing for years anyways). I'm now uncertain about whether I'll ever run any public groups, or post any public leaderboards, as they could have some negative effects.
The downside to exclusively private groups is it exacerbates the perception that indie game development is controlled by private cabals that hoard information from newcomers, but that can be balanced out by publishing tools that let people run their own groups, and by making it easy for people to join existing groups.
- Q: Are you saying that a game's only worth is what money it's earned?
- A: Not at all. There are plenty of wonderful and worthwhile games that are entirely non-commercial (I've developed several myself), and we all can list brilliant commercial games that sadly never sold very well. I started this project because I wanted to educate myself about how the market works and empower myself to survive its many hidden dangers; now I want to share that experience and toolset with all of you. Many of us are very interested in learning how to survive financially; that's the only reason we try to estimate sales.
- Q: Can't the results of SteamProphet predictors "move the market"?
- A: Definitely not at it's current scale. If it got really huge, then maybe, but I get the sense that there's so many players on Steam who don't even read blogs or watch youtube/twitch that we are probably pretty powerless to move the market in any big ways. There are a few effects StamProphet could have, however, and this is at the margins. The first is getting a regular stream of eyeballs on "hidden gem" games that would otherwise go entirely unnoticed. To be a good SteamProphet player, you need to look at every upcoming indie game. The second is that if anyone were to start gambling on SteamProphet picks at any large scale we could see new & weird vectors for scam games. This is one of the many reasons I don't allow gambling in any of my groups and I would strongly warn others against doing so in their own. The last one is that if we were to post SteamProphet predictions publicly, and SteamProphet started to gain a lot of recognition, a "herd mentality" might develop that becomes a self-fulfilling prophecy for games that are consistently looked over by top predictors. I have sincere doubts that we could keep destined hits from hitting, but I could imagine public leaderboards having a negative effect on smaller games and I want to tread carefully there.
- Q: I find SteamProphet to be insensitive for various other reasons, and/or am concerned that it could cause real harm.
- A: This project is still in its early stages so I would be more than happy to fully hear you out. Please email me at lars dot doucet at gmail dot com with anything you have to say.
- Q: Your rules suck because reasons! Can I change them?
- A: Hey, don't let me get in your way. If you're running your own group feel free to experiment with the rules and the scoring system.