The thing I have is randomly sampling days from the year however this runs into flaws of the day just not being optimal. Other research papers in related work has two years one for training and another for testing. I only have one year and asked the professor for another year of data and he said it isn’t available with no explanation behind it. I tried searching online for the data since it’s supposedly public data but can’t find it. It’s kind of hard to get an accurate display of evaluation when you don’t have a good test environment. Instead of complaining I have to figure out something. This is in the works in trying to get published to a smaller journal. Not sure if any of you all had this so curious how you would handle such situations?

At least an idea

  • Sample months instead of just time samples and create a pseudo environment and run it through there. Picking diverse set that has similar yearly trends.

I don’t have much experience so open ears when you all had similar limitations and how you all overcame it.

  • I_will_delete_myselfOPBEnglish
    1·
    10 months ago

    so, are you solving a markov decision problem here

    Yes. I am thinking of just using a metric to see if it made the optimal decision by the amount of value it delivers per capita.

    The main flaw from my previous metric is that it had a bias towards naive algorithms because the way its calculated which leads results to be misleading from reality. Skipping turns is sometimes the optimal decision which the metric said it was bad, but reality it isn’t.

    When I dug closer into the data it turns out the AI was destroying the naive algorithms with this metric and the total results we were aiming for.