"I am not exactly sure about Markov models, but Infer.NET is a great library for doing statistical inference." . . . .