This is a guide for using the RiskAMP SMART Conditional Risk web application, available here:
riskamp.com/smart/app/conditional-risk
We suggest opening this guide in a separate window so you can read it and follow along at the same time. You can download a completed version of the model used in this walkthrough here:
Revenue Projections 2016 model
Let's say that you are planning your next fiscal year, and you want to get a sense of your potential revenues.
You have a pipeline with some number of projects. You can estimate the likelihood that each project will go forward. You also have estimates for the revenue you expect from each project if it does go forward.
The conditional revenue model takes that data and uses monte carlo simulation to generate projections about the probability of reaching certain revenue levels.
The same model can be used to measure risk. Suppose that you are engaged in a project, and you can identify several sources of potential risk. You can estimate the probability of each problem occuring, and the cost it would incur.
In this case you can use simulation to project the probable level of risk in your project.
In each case, the benefit of using stochastic modeling is understanding what happens when multiple independent events occur or don't occur. In some cases, all the revenue opportunities might come through; in other cases, you might get some but not the others.
Combinatorial probability refers to combining multiple sources of uncertainty to understand possible outcomes.
In addition to modeling the base revenue probability, you might want to consider some additional possibilities. If you embark on a new marketing program, for example, you might believe that this would increase the likelihood of closing one or more revenue opportunities.
Or in the risk example, you might take some mitigation actions to reduce the levels of certain risks.
The simulation model can model these alternative scenarios, to see what the effect is on overall revenue or overall risk. You might then use this information to make a decision whether a particular marketing campaign, or risk mitigation strategy, was worthwhile.
From here on in we will talk about the revenue case only. But everything that follows will work exactly the same way with a risk model.
Here are a few useful tips before you get started:
Although this is a web application, all data is stored on your machine and does not get sent to our server. It's saved in what the browser calls "local storage".
We save the model all the time - every time you change it, in fact. If you accidentally close your browser, or navigate to a different page, you won't lose your data. Just come back to the application and it will be there.
You can also save your data to a file on your desktop. The File tab has buttons for saving and loading files.
If you make a mistake, press Control-Z to undo.
In the Conditional Revenue / Risk Model application, click the File tab. Click the Reset Model button to set a blank model. This will also take you to the Project Model tab.
First set a name for your model. Click the title "Unnamed Project", which will open an edit box. Type a name for your model, something like "Revenue Pipeline 2016". To close the edit box, press Return or click somewhere else in the window. All the edit boxes work like that.
The next step is to create the project items. One item is already created, so you can start by modifying it. First give it a name. Click the name "Unnamed Item", and type a name - in our example the first item is "Office Building". Then click the Edit link to open the item editor. The details are hidden just to clean up the interface if you have a lot of items, but you can open and close the editor at any time.
The next step is to set up the probability estimates for this project. First, the estimate for how likely this project is to go forward. Click under Probability to open the editor, and type "50%" (no quotes).
Next, if the project does go forward, we want to estimate the potential revenue. We do that with three estimates - low, high, and the most likely. Click under each estimate and enter a value. In our example, we esimate the low value at 500,000; the most likely value at 1,000,000; and the high value at 1,500,000. (Note that here I'm using US-English style numbers. Enter numbers however you would normally in your country, and the browser will understand). After you enter one estimate, you can press tab to jump to the next edit box.
Once your first item is set, check the Report tab to see what that looks like. If you used the values we listed above, you should see a pretty simple line graph. Check the tables to see some statistics about the model. As you add more items, the chart and tables will become more interesting.
Click back to the Project Model tab and at the bottom, click Add Item. This will add a new, blank item to the project. Name this second item "Shopping Center". Open the editor, and give it a probability of 40%. Then set the following estimates: low 1,000,000; most likely 1,300,000; and high 2,000,000.
Now click back to the Report tab. You should see the line chart now has some noticeable curves in it.
What that reflects is the possibility that you might go forward with one project, or the other, or both, or possibly none. The simulation uses 5,000 trials to test many possible outcomes. In each case, the revenue generated is also randomly sampled based on the estimates. The resulting chart summarizes the possibilities, based on the estimates you've created.
Click back to the Project Model tab and add one more item. Click Add Item at the bottom. Name the new item "Parking Deck". Open the editor and set a probability of 60%; then set estimates of 900,000; 1,200,000; and 1,800,000.
A big part of the analysis we do after a simulation involves percentiles. On the graph on the Report tab, values are marked for the 10th, 50th, and 90th percentiles. In the table we also include the P80 and P50 ranges, which are based on percentiles (more on these later). Percentile values are how we characterize risk and probability in the simulation model, so it's important to understand how they work.
When we run a simulation, the output is a large set of numbers — possible outcomes from the model. We want to organize this data so we can get some information from it. The first step is to sort the results by value.
Once the results are sorted, we can look at the middle value. This is the 50th percentile (also called the median). It's called the 50th percentile because 50% of the values are higher (or the same); and 50% of the values are lower (or the same). We know this is true, because we sorted the values and then took the value in the middle.
If you used the values we listed above, the 50th percentile value on the chart will be around 1,700,000 (it might be slightly higher or lower). In analytical terms, we can say that — during the simulation — there was a 50% chance that revenue would be 1,700,000 or more. Or put another way, there was a 50% risk that revenue would be less than (or equal to) 1,700,000.
The 10th percentile value in the chart means the same thing, except that it includes only the lowest 10% of values. The value is zero in the chart, which means that in the worst 10% of cases, the value was 0. In the simulation there was a risk of at least 10% that revenue was 0.
The P80 range refers to a range between the 10th and 90th percentiles. It's called P80 because 80% of the outcomes fall in this range. By using the 10th and 90th percentiles, it removes the most extreme values — the tails in the probability distribution.
Another way to describe the P80 range is to say, "this is what happened in all but the worst 10% of cases (the 10th percentile) and the best 10% of cases (the 90th percentile)". By removing the extreme values we try to get an idea of what might happen in most normal cases.
The P50 value works the same way, but it takes the central 50% — the range between the 25th percentile and the 75th percentile.
In addition to the distribution chart, which plots the value at each percentile, the report includes statistics about the results and some additional risk analysis.
The table showing the probability of reaching a target value is the inverse of the percentile analysis described above. Here, we are interested in a numerical value, and we want to know the likelihood of reaching that value.
The meaning of the percentile is the same. In the sample model, the probability of reaching 1,000,000 in revenue is a little over 80% (it may be slightly lower or higher for you). That means that in the simulation, there was an 80% probability that the total value was 1,000,000 or higher.
This would be the value if we took the 20th percentile as described in the last section. Because the 20th percentile means that 20% of the values were lower (or the same); and therefore 80% of values were higher (or the same). That means there was an 80% chance that the value was at least 1,000,000.
The contribution to variance chart (the donut) shows the sensitivity of the model result — the total value — to each of the individual inputs. It indicates how much of the variance in the result is explained by variance in the individual item.
What's the point of this analysis? There are two, basically. The first is to understand risk. The second is to use that understanding to make decisions.
Let's say that you are considering your spending plan for the next year. Based on the revenue projections in the model, you may want to adjust your expectations or your spending plan. If you believe that there's an 80% chance of revenue reaching 1,000,000, but only a 12% chance of reaching 3,000,000, will that affect your spending plan?
If you are planning financing for next year, would this revenue distribution information affect the level of financing you seek?
Decisions on spending, financing, organization planning, and the like have to be based on many different factors, but getting an understanding of how probabilities interact can help by providing some visibility into possible outcomes.
The next step is to consider what might change in the model in the event of changes in circumstances. Here we'll consider the effect of a new marketing and sales effort. The same analysis might apply if you wanted to understand the effect of changes in the economic climate; or in a risk model, if you wanted to undestand the effect of taking some risk mitigation actions.
Click back to the Project Model tab. Open the editor for the first item if it's not already open by clicking Edit.
At the bottom of the editor, click Add Alternate Scenario. This will add an alternate scenario which will reflect our estimates in light of changing circumstances.
After you have added a new scenario, click the title "Unnamed Scenario" to edit it, and enter the name "New Marking Plan". You can also re-name the first scenario. In our sample model we've named the first scenario "Current Forecast".
Now fill in estimates for the first item ("Office Building") in the new scenario. In our updated marketing plan, we think the estimates for low, high, and most likely revenue will be the same. But we think that we can increase the probability of closing this revenue opportunity. Set the probability estimate to 70%.
Edit the next two items as well. The new scenario is automatically added to each item. If you want to leave an item out of the model (it will never happen under these circumstances), just set the probability to zero.
In the second item ("Shopping Center"), set the probability estimate to 60%. In the third item ("Parking Deck") set the probability estimate to 80%.
Now click over to the Report tab to see what the new scenario looks like.
The chart will now have two lines representing the two scenarios. You'll see that the new scenario has higher values at the various percentiles (as you might expect, since it's more optimistic). But it also has a different shape than the original line, because the probabilties interact in different ways.
In the Statistics table, you can see that the median value (the 50th percentile) is considerably higher, and the P80 and P50 ranges are significantly improved.
In the next table, "Probability of Reaching Target Value", you'll see that the probability of reaching 2,000,000 in revenue jumps from (roughly) 46% to 74%.
At the bottom of the report there's a new chart comparing values between the two scenarios at different percentiles. Whether this is positive or negative depends on the order of items in the chart (change the select boxes to see what I mean).
The chart is calculated by taking the value at various percentiles from each scenario, and subtracting. You'll see that for the most part, revenue under the new scenario is higher by 400,000 or more — sometimes a lot more.
With the alternative scenario, we have another piece of information we can use in our decisionmaking. In this case we were modeling the effect of a new marketing plan. We saw that, during the simulation, the new scenario returned significantly more revenue — at least 400,000 more, except at the very high end.
If the cost of this new marketing plan were 300,000, would this information influence your decision to implement the new plan? What if the cost of the marketing plan were 600,000?
In this example we only modified the probability of each item, but in other models we might vary the expected revenue (or risk cost). If we were considering changing economic conditions, for example, we might increase or decrease the revenue estimates along with the probability. That would have different effects on the probability distribution.
The example files on the File tab show a few examples of alternate scenarios with different probability characteristics.
It's important to note that this kind of model is entirely dependent on the quality of the estimates you can provide. The classic way of describing this is "garbage in, garbage out". If your estimates are wildly optimistic, your projections will be very good, but they won't have any basis in reality.
When using this type of analysis it's very important to carefully consider how you construct your probability and revenue estimates. You can always make changes as conditions change, to see how that impacts your overall projections.
It's also important to understand probability. Humans are typically not very good at this; we see a bet with a 90% probability and think it's a sure thing. It's not — it will lose 1 time out of 10, and that might be the time we play.
All this type of analysis can provide is information. It's up to you to balance that information with information from other sources, and with your own tolerance for risk.