Hello everyone, I hope this subreddit is the right place to seek help!

I have a system with multiple states (N) that can transition from one state to another at every discrete time increment, or stay in the same one. I want to obtain a good estimate of the transition probabilities of the system.

I have some data that allows the creation of a transition matrix, treating the problem as a Markov chain. However, there are extra covariates that I would like to use to further “segment” the states. By doing so, I may end up with quite little data, and I’m not confident enough that I would be able to represent the actual system accurately.

One solution I thought of was to create a multinomial classifier that, given these extra covariates, provides a probability for each (next) state. However, I find it difficult to evaluate the goodness of such a model, as there is no good metric to evaluate the entire vector of probabilities that the model will provide for each single combination of covariates. In a normal classification problem, I would look at metrics like accuracy, recall, or precision based on the nature of the problem. Here, I am interested in ensuring that each predicted probability for each state is accurate, making things a bit more complicated.

To address this, I was thinking of using a more Bayesian approach, but I’m not sure if it’s actually Bayesian or if it makes sense at all. The issue of small data makes any particular estimate (in the sense of covariate combinations) not that reliable. However, I would be fine providing a transition matrix with ranges and not “absolute/expected” values. To do so, I was thinking of sampling M times without replacement from a smaller portion of the data (say 80%) and creating, for each combination of covariates, M possible matrices. For each entry, I would provide the expected value plus or minus the standard deviation, assuming that those values are normally distributed.

Here are my specific questions:

1. Would the proposed solution make sense?
2. If yes, how do I establish the percentage of the data?
3. Is there a better solution?

Thank you in advance for your time and brainpower! :)