How will ML be implemented?

Machine learning will be implemented to analyze and predict potential player results based on a specific set of criteria. i.e. a fantasy league player roster.

Which models will you implement and why?

• Linear regression: To simulate a mathematical relationship between variables such as goals, assists, and points to predict a total score outcome.

• Neural Networking: For learning patterns in the data to predict trends such as whether a player historically plays better or worse against one specific team than others.

• Naive Bayes: For making real-time predictions based on changes to a fantasy league players roster.

How will you measure performance and why?

• Accuracy: The ML will be measured based on how well it predicts a winning roster or not. This is important as it is the main purpose of the ML.

• Precision and Recall: While the ML provides a prediction, there is still a human factor it can not pull information from. Understanding and comparing the difference between what the ML predicts and what actually happens will be important to continuously train the model.

• Processing Speed: The ML will be measured on how quickly it can assess the data and provide a result/prediction. This is important because the purpose of the ML is based on a pre-determined timeframe i.e. a weekly competition. If the ML takes too long to process and can not provide a result in time for the competition, then the ML is not useful. 

Will your ML system be cloud-based, or edge-based, and why?

The ML system for the NHL fantasy coach would need to be cloud-based as the high volume data collection would need to be supported and leveraged from one single source, then pulled into a profile based on user roster. In addition:

• There would be the potential for better accuracy in predictive analysis - pulling data from the same reference source as competitor data.

• Concern for privacy and security leak is low since most of the data collection will be publicly available.

• Real-time learning is not important since users would have already selected their rosters before a game has begun -- so the need to reduce latency is very low.

Will your ML system implement batch learning, or online learning, and why?

ML system will implement batch learning. This is because:

• The model needs to retrain itself once a day - after games have been completed versus during a game.

• The roster of a fantasy league competitor may change and updates would need to be made to the "dashboard" so that a new predictive analysis can be generated. This process could be managed through online learning but the cost and frequency of use would likely not support the ROI.

Will your ML system implement batch prediction, or online prediction, and why?

ML system will implement batch prediction. This is because:

• Prediction analysis can only take place before or after a game and thus does not need to be in real-time.

Which issues are High Impact, Urgent; High Impact, Not Urgent; Low Impact, Urgent; and Low impact, Not Urgent?

High Impact, Urgent:

• Data Collection: Ensuring data that is being pulled into ML is coming from a reliable source. This would need to be confirmed against other data sources for confirmation.

• Model Optimization: It will be important to train the model quickly so that it can analyze and produce results based on an informed ML.

High Impact, Not-Urgent:

• Transparency of ML and Data Collection: Letting users know how the data is being pulled, from where and how it's processed, adds legitimacy to the product. It will be important not to share too much so as to risk unique selling features.

Low Impact, Urgent:

• N/A

Low Impact, Not Urgent:

• N/A

Compare essential aspects of requirements The Deon Checklist.

• Data Security: Ensuring user personal information is safe, secure and doesn't include any information like street address or location - especially since this information is not relevant to the product purpose.

• Right to be forgotten: Ability to remove users from the database - should they wish.

• Data Retention Plan: While historical player data may be important, keeping user information year over year may not be. Users who do not sign up within 3 "seasons" would have their profiles deleted.

• Dataset Bias: ML needs to be constantly retrained so as not to favor one consistent outcome i.e. over favoring a player metric that seems to overrule others unfairly.

How will you resolve any potential issues from the comparison above?

• Ideally taking a preventative approach versus a reactive/resolve approach.

• As we build out the PRD, we would include a section on Ethics linking to the DEON checklist and ask the question, "Could this be used to hurt someone? If yes, how and how can it be prevented or governed to mitigate this from happening?"

• In the case an issue does become apparent, we would roll back the product or feature and iterate accordingly to resolve the issue (if possible) before deploying again. If the issue is severe, the feature or product may need to be sunsetted and not deployed again.