How we prioritize
Rasmus Rønn Nielsen, The Dudes
15 May 2012, 12:59
As you know, there's literally hundreds of ideas for improvements we'd like to make. But we haven't got time for it all. That's why it is very important, that we choose the right ideas to work on.
So what are the right ideas? It's the ideas that improve your overall experience of the game the most while not taking too long to develop.
It'd be cool if we had photo realistic 3d matches in Virtual Manager's match viewer. But if it takes 10 years to develope, it might not be the right place to put our efforts.
Unfortunately, it is extremely difficult to estimate the level of amusement and development time required for a certain feature idea. An idea that sound like fun, might be boring in practice. A feature might seem like something we could develope in two days, but perhaps it ends up taking months.
To become better at these estimations we make prototypes regularly. A prototype is just a quick and simple implementation of an idea. Quality-wise, they are not good enough to be added to the game. But helps us understand how fun the idea is, and how long it would take to develop a proper version.
The prototype for our mobile app is a great example:
It is unlikely that this implementation in particular, will ever be completed. But we learned a lot about developing mobile apps. And we got better at estimating the development time of this kind of software.
So prototypes makes us smarter - we become better at prioritizing. When sufficiently confident, we choose a project to make "for real". On top of being able to choose the right thing to work on, we can now avoid making the same mistakes we did with the prototype.
It sounds troublesome, but it's worth it. Because it means we can deliver the right and the best improvements for you :-)
PS: For the last couple of months, we've been working on a new big improvement to the game. We'll tell you more about this in the next blog post.
Comments
when will your next blog be?
Ok where the next stage in the stands mmmm mmmm
Promise it be soon
Will be man says pigs may fly
Vm said hot dog stands soon
Which is truth which is not
I might belive more chance pigs may fly
Sounds like a lot of excuses if it takes too long to do the big things implement the little ones first like naming stands. reomoving auctioned bids which you are not leading if you wish etc. Before match experience and loans '''for example as bigger things)
So let's say replica shirts generate 10c per supporter and 90% of supporters buy them. Its not an amazing amount of income but depending on the clubs size it could help generate income for the club and bring a little excitement to the game.
It might be more reasonable to estimate the income of club shops at the end of each season. So for example a small club with 300 supporters would sell about 270 replica tops and 10c each, earning 2,700c income on replica tops alone. Add other expenses like club programmes. Sold per game with just 1c each. Estimate a rough percentage of sales per average attendance (say 400 out of 1,000 in attendance per game). With 15 home games and 400 sales per game, that's 6000c per season added to income.
Burger shops and general food stands, again if we assume 40% of attendence gets food in the game that's another 6,000c
So for a small club like mine, by investing in a team shop I could earn 24,000c per season on top of my ticket sales.
Looking forward to seeing the next improvement though
- You have a half finished stadium editor in which you promise us a "part 2". So far no one has seen the benefits of this, but you promised us that when part 2 arrives we will see how its better.
- You then start work on a mobile app . This will not benefit all as not everyone has an iphone
- Now you announce a new improvement which may or may not be related to mobile apps and stadium.
Then you make a blog telling us about how to prioritise?
Seems like we need to be telling you.
Really the only thing that isnt possible to do on there is placing players in the tactical setup, as drag and drop is not possible,
So at this point there's no mobile app in the App Store and there probably won't be for quite some time. We do want to make an app, but we can only do so many things at a time.
I wrote that "this implementation IN PARTICULAR" won't be completed. That's not insinuating that we won't make an app. We do like to make an app. Of course we will, it would be awesome to have that.
But we're small team and we have to prioritize. Making a prototype and realizing this isn't the right time for an app is not "an excuse". It is thoroughly well-based prioritization.
This was exactly what I tried to express in my blog post. Just because we don't constantly deliver new features doesn't mean we're not being productive. To us, it is more important to make great things than just making things.
Things that are not great is not shipped.
Three guys producing a photo realistic 3d simulation of a football match in four years? I hate to be negative, but that's just isn't possible. At least not in this decade :-)
Even EA Sports with lot of programmers, 3d artists, designers and a huge budget cannot make truly photo realistic graphics yet.
Instead, we're considering prioritizing making tactic editor and training instructions work better on mobile devices. That'd be a start at least...
It was just "this implementation in particular" that won't be completed.
At the moment, I cannot say when we'll start producing the real app.
But that doesn't mean we are not listening to you guys. We are aware that youth academy is much wanted, and we are going to respond to this at some point.
The proposals list isn't dictating our prioritization but it is certainly affecting it.
I find it mildly disturbing that you feel in position to conclude that "no one has seen the benefits" of the stadium editor. The fact is we saw an increase in the number of new users when we released it. And we've gotten feedback from a lot of people that they really like the new visual expression.
The number of paying users also increased.
So saying that no one sees the benefit of the stadium editor simply is not the truth. What I think you mean is that most experienced users generally don't care. But that is another matter.
An app was in high demand and we were pretty sure this need would not be getting smaller anytime soon. So I cannot see anything wrong with us getting our feet wet in this area. We did by experimenting with a real product. The best way to learn. We never told that this product would be released because we were not sure. After this experimentation we were confident to say that the product simply was not good enough.
And we learned that we cannot make an app "just like that". A great app would take more than a year to complete. And in this perspective, we assessed that we currently could focus elsewhere to generate more value in less time.
What would you suggest we do? Magically make it the right way the first time around? We are human, so we cannot make anything great unless we practice first. So that's not an option. We could of course, also have skipped the "exercise" altogether but then we would have ignored a very popular request from our users. How is that better?
The same goes for the stadium editor version 2 by the way. It would be great to have, but when you take development time and increased "funness" (in a lack of a better word...) into account we had more valuable things to focus on.
And just to be clear, I'm not claiming we did this the perfect way. But we did it in a very rational way carefully weighing out all pros and cons.
Sure, we'd always like to hear how you think we should have prioritized
It worked OK, but it just didn't feel as nice as a native app.
That's probably the hardest question we face onwards: Should we create a decent web based experience for all mobile platform, or a single great native experience for only one platform?
Advantages and disadvantages to both approaches...
While VM has collected hundreds of thousands in subscription money this year, and we're over a third of the way through the year, there hasn't been a single improvement made to the game in 2012.
Is there not a way of changing the way these two actions are completed so that we can have full functionality for now until the app is available?
I do not have a clue about programming or code, but this surely would be a quick fix to keep people happy for now??
+whats gonna be like?
and how do you get a logo?
:-) !
You mean you develop what makes you money.
Just say it! Don't be shy. You bore us with questions about what we want and then do something different.
You aren't interested in us and the game. You are interested in what gets you cash. And as a business that is fine. Just don't pretend it isn't your top priority!
And penny is right about the stadium editor. It was an awful half thought out update!
I pay for this game,...i wont be using the app. So my money is going towards something i wont benefit from.
definitely not renewing premium membership.
Waste of time.
They know what they're doing - people need to leave them to it.
Simple reason being that the more money teams have the more they will spend on players.
Its called inflation, players will become more expensive by about as much as any additional revenue streams, as described, will create.
None of this will improve any individual team, nor will it add to the game play...
Therefore it will almost certainly become a reality in due course.
The part 1 of the stadium editor was released 6 months ago... wouldn't it have made more sense to follow up on that? I think not, because the form (pretty picture) was more important than the details and the un-thought of economic effects (read old blog posts for proof that Rasums didn't understand it himself)
I think anybody who thinks the match viewer will ever be 3D graphic is either deluded, - or far more likely - simply too young to actually grasp what that would entail.
It's because the good things about this game were not about graphics that the stadium editor is such a fiasco. Sure - young kids love the pretty picture, because there is an inverse relationship between liking the picture and understanding how much the cost of expansion actually prohibits advancement.
But they'll still not feel right. And that's important. Just because you package your web app inside af native webkit view doesn't mean it hasn't got the performance implications related to a web app. The finale product just won't feel as good as a real native app would have.
On top of this comes all the concern of resolution size and ratios of each platform. Also, the design guide lines are totally different across platforms. A navigation system that appear intuitive to an iPhone users might be confusing for an Android user.
I hate to be negative. But the fact it is, that it is not as easy as just using PhoneGap and everything will be good. Unfortunately.
I recommend these blog posts on the subject:
http://sandofsky.com/blog/shell-apps.html
http://rrn.dk/mobile-apps-web-or-native
Also, planning, experimentation, playing and learning takes up a great deal of time. It is required in order to produce great products.
You might argue that we have not published enough new functionality. You are fair to have that opinion and you are free to leave the game if you feel others do a better job than ours. But saying that there hasn't been a single improvement in 2012 is nonsense.
We have examined the case and it turns out a "quick fix" isn't really plausible unfortunately. However, everything new we make (including the new tactic editor coming with the new match simulator) will support touch devices 100%.
In our latest blog post you can read more about what we believe is best for the game.
But we're also trying to create games.
We unify this in the belief that focusing on creating great user experiencing is the best way to create long term profitability and user satisfaction.
In time, the profitability will allow us to make even greater games.
You might agree with Penny about the stadium editor. But the fact is we got more paying subscribers than ever before during that period. Also, the number of unique logins had the highest percentual increase in several years. So insinuating the stadion editor update was a bad thing is just plain wrong... unless you want the game to have fewer happy users.
Instead, we do the second best: trying to makes a many as possible happy. And right now, a lot of people want an app. And that's why were considering fulfilling that need.
Right now, we feel it makes most sense to put all efforts into making a new match simulator.
You can read more about why did not prioritize stadium editor version 2 in our latest blog post.
Please elaborate on why I did not understand the economic effects. I'm curious and would like to defend myself, but it's near impossible when you're not being concrete about it.
In fact, the stadium editor was the greatest success we had in two years. At least if you measure in number of customers and sign ups.
The stadium editor was a bad update, i'm not saying that it will have stopped new subscribers but i am saying that it caused the highest ever excedous of existing users that i have ever seen!
The way it was sold to us wasn't correct. It was never the users idea. It was an idea you guys had to slow down the divide in credits between top and bottom for the longevity of the game to ensure you had a game for users to subscribe too.... infact top clubs leaving does you a favour because it brings the divide closer.
Personally i think there are easier ways than that, but its your game and your choices.
Just don't give users false hope.....
The percentage increase of both numbers where higher when we released the stadion editor that it had been for years. So in our mind, calling it a bad update (some have even called it a "fiasco"
Sure, some were affected. But that can't be avoided when changing the rules of a game.
The stadion editor was the top most requested feature on the surveys we sent out. We did several times and each time there was one clear favorite. The majority wanted the stadium editor. We met the demand, and created it.
I'm not sure what you are suggesting here, but we weren't trying to change the longevity of the game with stadium editor update. We tried to creating a better overall game experience in which you could actually visually see your stadium and the parts that comprises it instead of just looking at borring text.
Making this change had some implications on how the updates were performed, and we did our best to convert the system to the new style in a manner that would yield the best playing experience for the user.
I'm not sure I understand you correctly, but it seems you are insinuating that we deliberately were/are trying trick our users. If my assumption is correct, I'm really sorry you feel that way. We want to create the best user experience and tricking our users aren't apart of that.
I don't agree however, that the new stadium system is a step backwards in itself. Our numbers indicate the highest increase in new users and new paying users in several years around the release of the stadium system.
But the fact that you, and many others cannot upgrade your stadium is a problem that should be fixed. I just don't think it is our top priority right now.
And yes, we are aware that some were hit but that's often inevitable when changing the rules of a game.
Well of course them believe. At least I hope they do. I hope that ALL our users believe what we say. Why would we lie to our customers?
What part of what we offered didn't we deliver on? I really want to know, because if you're right, we have some rectifying to do.
I'll do my best to make sure we find time for working with "the stadium issue" but I'm afraid I cannot promise anything. We have a lot of users and each of them want us to work on something different, not easy making everyone happy.
Thanks for still being around though
But you have now said that you don't know where to start or what to do with the Youth Academy? So what is the point on getting people to vote for something that just can't happen.
You would be best saying, right, we can do a, b and c... so lets vote.
I think if you were to run a poll today and ask users to vote if they believe that any of these upgrades are going to happen, then you would get a 60/40 split in favor of NO.
Are you saying that we should spend our time whatever has the most user votes? If so, I don't agree it would be the best way to improve the game.
I elaborate on our point of view on feature prioritization in this blog post.
You might be right about the going-to-happen-poll.
I think you should work on what is achievable. And what ever makes you the most money.
One of the things we also consider is user excitement. And the proposals page is a great way to get that information. Without it we would be blindly guessing what people want.
I hope this explains why we get people to vote.
However, it doesn't make sense to just make the most voted for feature without regarding things such as required development time, achievability, accessibility, usability, level of entertainment.
We have to consider them all. So, just because we don't blindly make whatever is at the top of the proposals page doesn't mean the information doesn't have an impact on our decision.
(1) I said that there hasn't been a single improvement made to the game in 2012
(2) I said that Even Rasmus didn't fully understand the economic impact of the Stadium Editor.
Rasmus refuted both of these points. Re (1) he said "We've made hundreds of small improvements and fixes" in 2012.
First of all, Fixes do not constitute improvements. It's like a rail network saying that fixing a broken track is an improvement. Fixes are run of the mill, and are expected from customers. There should be little fanfare about fixes.
In terms of improvements: Rasmus mentioned an improved cheat hunting system (incidentally, those who get away with cheating now - post Stadium Editor economy - benefit far more in relative terms than they did before the Stadium Editor. More of this later. Great, although there's still repeated posts on the forums about cheat-hunters abusing the system and punishing those who have done little wrong.
If there have really been 'hundreds' of improvements, you'd think users would be identify at least 1.
But in the post below, nobody could identify a single improvement made this year.
http://www.virtualmanager.com/forums/topics/953473-new-to-virtual-manager-in-2012
Maybe changes had been made without communicating them, but if some actual improvement had been made - better tactical options, change in the match engine, an end to the stupid **** of sendings off / injuries etc.. even graphic/layout changes that people thought benefited... I'm sure I, and other users (at least one user viewing the forum) would have piped up and spoken of such improvement(s).
Re (2) Rasmus said "Please elaborate on why I did not understand the economic effects. I'm curious and would like to defend myself, but it's near impossible when you're not being concrete about it."
Here goes: In Blog posts during and after the launch of the Stadium Editor, Rasmus said the following:
"Admitted, some larger clubs will see a slight decrease in their profits, but it has become equivalently more expensive for smaller clubs to reach their level (in terms of stadium capacity) because seats have become more expensive for larger stadiums.
There is no denying that this change will affect most clubs' finances. If we look only at the profits generated from ticket sales, small clubs will likely see a little increase while larger ones will see a minor decrease."
Larger clubs - those with 50,000+ seater stadiums - would have suffered around a 30% - 40% decline in income from ticket sales - even AFTER putting ticket prices up. I was in the top division, and if I put my ticket prices up by 2 credits, I wouldn't fill my stadium. A 30%-40% decline in income from stadium revenue is by no means a "slight decrease" in profits. Seems like a gross mis-understanding of the economic impact.
Further, while profit from ticket sales drops by 30%-40%, wages have stayed the same, as has transfer prices - which is driven by supply and demand... There is still enough demand for good players (young or old) so prices haven't dropped by 30%+.
If I want to upgrade just ONE of my stands from level 4 to level 5, the costs are prohibitively high.
It would give me 4800 extra seats, and would cost me 18,980,000 (rather than the c.5,200,000 it would previously cost me). THE PRICE OF AN UPGRADE AT THIS POINT IS OVER 3.5 TIMES MORE THAN IT WAS PREVIOUSLY.
Under the old pricing, where it would 'only' have cost me 5.2 million to upgrade, I would break even after about 9 weeks. That is to say:
By upgrading 4800 seats, I will make an extra 33,600 a match based on mid point ticket price of 7) - (4800 x 7 = 33,600).
This means it would take 155 games before I made back the 5.2 million I spent upgrading (155 x 33,600 = 5.2 million) . Assuming 17 home games a season (15 league, 2 cup), it would take 9 weeks to reach these 155 games, and break even.
After that the additional 33,600 per match is all profit (around half a million a season), which is why it was worth upgrading, since extending this further, an additional 10,000 seats = an extra million in profit a season once you've paid off the investment.
Under the new system, instead of taking 155 games - or 9 weeks - to break even and start making a profit on your investment, it will now take over 560 games - 33 weeks (over 7 months) before you break even and actually start profiting (at exactly the same level you would have before) from your upgrade.
Far more expensive to upgrade, less revenue from tickets, means it is far, far harder to advance in the game once you've been playing for a year or so. Rasmus has pretty much admited as much in his response to Bath Rovers:
Rasmus: the fact that you, and many others cannot upgrade your stadium is a problem that should be fixed. I just don't think it is our top priority right now"
- This is a "Problem" that should have been apparent before launching the Stadium Editor...All it required was some simple maths. Again, this looks like Rasmus failing to understand the economic impact.
Finally, Rasmus said" Some managers are nervous that this might give the clubs who trade a lot an advantage since their income is not directly hit be this change. We're sure they won't. The money that these so-called "trade clubs" earn come, when all comes to all, from the same source all other Credits come from: ticket sales. A certain reduction of the general production of Credits will thus hit the trade clubs just as hard as clubs that do not trade so much. "
- This is to completely misunderstand how traders make money.They make money by being online more, and being able to pick up players on auction , and sell them on at a profit (either on auction during busier periods, or via the transfer list, where prices are always higher).
Even if profit margins were cut slightly, traders will still be able to trade and make a profit. Proportionally, - and relatively, and compared to before the Stadium capacity reductions - the amount of money traders make will be greater than the amount people make from their reduced capacity stadiums. I.e. the exact figures will differ by each person, but as an example, previously a trader might make 30% as much as I was earning through ticket revenue... now, with the stadium reduction, they might be making 50% as much, because my ticket earnings will have gone down by far more than theirs has...
Trading is about supply and demand and time on line and that's not going to change. Again - this looks like Rasmus misunderstood the economic impact the stadium editor would have.
Edit: While large numbers may have joined to see the new Stadium Editor, there's currently around 1500 people online vs. the 2500 or so that used to be online after the Sunday updates, so not sure how many of these new recruits have stuck around / continued to pay for VM....
Making customers happy is very challenging. I do appreciate this. Particularly in this environment where there are a lot of younger players with unrealistic demands (3D matches) and expectations. But players have been saying they want improvements to gameplay, yet, in your end of 2011 summary, you said you'd be concentrating on graphics improvements... I'm glad you say you're now going to look at the match engine, as this is something clear and tangible that could / should improve the game greatly...
I agree fixes don't constitute improvements. That's is why I wrote "improvements AND fixes". I also agree there shouldn't be a lot of fanfare about fixes. That's why we usually don't spend time blogging about it. We've got more worthwhile things to do.
The critique about abusive Cheat hunters is based solely on rumours and therefore not worth much. The essence of the issue is, that Cheat hunters need to have certain extra tools in order to do their work. Some people find this disturbing. But the alternative is to have no CHs at all. Still, we've listened to the critique and improved the system to mitigate the benefits of using their special tools selfishly.
I didn't say we made 100s of improvements. I said we've 100s of improvements and fixes.
I think our disagreement is due to a different understanding of what "an improvement" is. It sounds like you feel an improvement is something that actually changes the feel or the gameplay of the game in a very noticeable manner. In my mind, an improvement can also be a small improvement.
During the last week, we have made these improvements:
On top of this I suspect we've fixed 10-15 bugs.
And all this while in a time where we're actually spending most of our time on the new match simulator.
Throughout 2012, the number of actual improvements and fixes have been a lot higher than this "per week". This is because we are not always focusing on a "big new feature" like we are right now, and that makes more time for general improvements.
Together with all the bugs I am absolutely sure that are several hundreds in total.
I'm not saying we've done everything perfectly. We haven't - that's for sure. I'm not even arguing that we are very productive (we're trying to be). I'm just arguing that we in fact have made several hundred fixes and improvements throughout 2012.
Regarding your comments about the stadium editor's effect on the economy, I must say you have some valid points. We should've done better in anticipating how the change would affect the economy. 7 months break even period is just ridiculous. If this truly is a general problem, we should've seen that coming.
However, I don't agree with your reasoning about the market prices and the traders. You can't both have a global decrease in profits and still see no change in player market values (as you suggest). That doesn't make sense to me. If people profit less, they have less money. If they have less money, the amount they'll be willing to pay for a certain type of player decreases. And this means the player values decrease.
And if the player values decrease, so does the absolute amount you can earn from being a trader. I'm not saying traders won't continue to have an advantage. They will, and that's a good thing. But a overall decrease in profits will inevitably lead to a decrease in player values which in turn will lead to a decrease in the absolute profit from trading. It is easier to score 5,000 Credits from a player that is valued at 100,000, than one that is valued at 75.000.
So, either the player values are unchanged (which suggests the new stadium editor haven't affected profits much globally), or they have decreased which in turn, as explained above, have increased the absolute amount that can be earned from trading.
Throughout the last two years, the number of people online have been decreasing. But firstly, that doesn't necessarily mean that fewer people play Virtual Manager. Read more about this in our blog post from last July. Secondly, the number of online users has been very stable since we released the stadium editor so that argument isn't really viable when discussing how the stadium editor have affected the overall popularity of the game. I can provide proof for this if you don't believe me.
Based on feedback from you and others, we've learned that we should prioritize performing a recalibration of revenue/expense ratios for clubs in certain "layers" of the game. We've already done some work in this area and that involves increasing the number of playable countries dramatically and by that making a more "flat" spread of the global revenue stream. The idea is that in this way we can keep the improved price structure of the new stadium system while still resolving the revenue/expense ratio problem. But that's a work in progress.
Thank you for your persistence and for taking time to compose such wonderfully articulated and well-founded critique.