# Creating a Probability Transition Matrix

Page 1 of 1

## 2 Replies - 1395 Views - Last Post: 28 December 2012 - 10:15 PMRate Topic: //<![CDATA[ rating = new ipb.rating( 'topic_rate_', { url: 'http://www.dreamincode.net/forums/index.php?app=forums&module=ajax&section=topics&do=rateTopic&t=305093&amp;s=c62484434ee8c706c98c7fbc86374257&md5check=' + ipb.vars['secure_hash'], cur_rating: 0, rated: 0, allow_rate: 0, multi_rate: 1, show_rate_text: true } ); //]]>

### #1 Lavace

• New D.I.C Head

Reputation: 0
• Posts: 7
• Joined: 03-January 11

# Creating a Probability Transition Matrix

Posted 28 December 2012 - 09:00 PM

I am trying to implement the following algorithm:

In a nutshell, we iterate over a set of 'states' to calculate a value for each.

Each state has multiple actions, and each multiple action has a 'transition probability' associated with it, such that if an action a is taken, there is a 0.8 chance it will proceed to a state s', and a 0.2 chance it will proceed to a different state.

When working through the algorithm, I am struggling to make a data structure that allows to locate a state, have a multiple actions available within that state, and for each action of that state, a probability matrix that when a particular action is taken to move to a new state, returns the probabilities of moving into a new state.

Any ideas to help me?

Many thanks.

Is This A Good Question/Topic? 0

## Replies To: Creating a Probability Transition Matrix

### #2 macosxnerd101

• Games, Graphs, and Auctions

Reputation: 11449
• Posts: 43,165
• Joined: 27-December 08

## Re: Creating a Probability Transition Matrix

Posted 28 December 2012 - 09:07 PM

A decision tree or a graph sounds like a good data structure to me. Perhaps a directed graph, specifically. You could also design a probability matrix using a 2D array or a HashMap.
Was This Post Helpful? 0

### #3 macosxnerd101

• Games, Graphs, and Auctions

Reputation: 11449
• Posts: 43,165
• Joined: 27-December 08

## Re: Creating a Probability Transition Matrix

Posted 28 December 2012 - 10:15 PM

Thinking about this some more, you should look into Markov Chains with Eigenvalues and Eigenvectors. This is a topic in Linear Algebra.
Was This Post Helpful? 0

Page 1 of 1

 .related ul { list-style-type: circle; font-size: 12px; font-weight: bold; } .related li { margin-bottom: 5px; background-position: left 7px !important; margin-left: -35px; } .related h2 { font-size: 18px; font-weight: bold; } .related a { color: blue; }