The Cosmetics Forum Forum Index
The Cosmetics Forum
The Cosmetics Forum is the place for skin care, hair care, makeup and other beauty product discussion.
Reply to topic
Model Validation on Very Small Test Data Sets
JoeClark


Offline

Joined: 22 Jun 2017
Posts: 117
Reply with quote
Hello,
This is very much an applied business/predictive modeling problem.
Typically when I have developed predictive models in the past I have trained on a large sample of data (i.e. n~10,000) and performed validation and testing on smaller out of sample data sets (n~3000) that are representative of the size of population I will ultimately score in a production environment. If model performance holds across these data sets, and if in production the model continues to perform well, I feel that I have a useful model.
Typically my models classify subjects into different risk pools. They are not perfect. Sometimes a few people in my 'low risk' group will experience an event, sometimes a few in my high risk group will never experience an event. However, these 'misclassifications' if you want to call them that are within an acceptable (practical) range of tolerance in the production environment, which again involves scoring 'live' cohorts (n~3000).
Thanks


I didn't find the right solution from the Internet.


References: https://communities.sas.com/t5/SAS-Data-Mining-and-Machine/Model-Validation-on-Very-Small-Test-Data-Sets/m-p/138008


Datacenter Management Video


Publicité
Publicité


PublicitéSupprimer les publicités ?
Model Validation on Very Small Test Data Sets
All times are GMT + 1 Hour  
Page 1 of 1  

  
  
 Reply to topic