Is it ever recommended to use mean/multiple imputation when using tree-based predictive models?Orthogonal sets of variables in multiple imputation --> separate imputation models?Multiple Imputation Using Different Data Setsusing cluster information in multiple imputationMultiple Imputation for Spatial Modelsmultiple imputation models containing categorical variablesWhen to use multiple imputation chained equations vs regression to impute data?Multiple imputation when explained variance of imputation model is lowPredictive Mean Matching as Single Imputation?How to apply a model built using Multiple Imputation to predict on dataset with missing data?How NULLs in numerical variables are treated in tree-based models?
How to make healing in an exploration game interesting
Are all passive ability checks floors for active ability checks?
If I can solve Sudoku, can I solve the Travelling Salesman Problem (TSP)? If so, how?
Light through a Sound Wave
The meaning of 振り in 無茶振り
Unnormalized Log Probability - RNN
Simplify an interface for flexibly applying rules to periods of time
Could the Saturn V actually have launched astronauts around Venus?
Have the tides ever turned twice on any open problem?
Why did it take so long to abandon sail after steamships were demonstrated?
Violin - Can double stops be played when the strings are not next to each other?
What is the term when voters “dishonestly” choose something that they do not want to choose?
Print a physical multiplication table
How could an airship be repaired midflight?
How are passwords stolen from companies if they only store hashes?
What is signal ground
What is the relationship between relativity and the Doppler effect?
Do I need life insurance if I can cover my own funeral costs?
Should Stotras and Mantras be recited aloud?
Should I use acronyms in dialogues before telling the readers what it stands for in fiction?
World War I as a war of liberals against authoritarians?
I got the following comment from a reputed math journal. What does it mean?
Did Ender ever learn that he killed those two boys?
Is it insecure to send a password in a `curl` command?
Is it ever recommended to use mean/multiple imputation when using tree-based predictive models?
Orthogonal sets of variables in multiple imputation --> separate imputation models?Multiple Imputation Using Different Data Setsusing cluster information in multiple imputationMultiple Imputation for Spatial Modelsmultiple imputation models containing categorical variablesWhen to use multiple imputation chained equations vs regression to impute data?Multiple imputation when explained variance of imputation model is lowPredictive Mean Matching as Single Imputation?How to apply a model built using Multiple Imputation to predict on dataset with missing data?How NULLs in numerical variables are treated in tree-based models?
$begingroup$
Everytime that I am making some predictive model and I have missing data I impute categorical variables with something like "UNKNOWN" and numerical variables with some absurd number that will never be seen in practice (even if the variable is unbounded I can take the exponent of the variable and make the unknown values negative).
The main advantage is that the model knows that the variable is missing, which is not the case for say mean imputation. I can see that this could be disastrous in linear models or neural networks but in tree-based models this is handled really smoothly.
I know that there is a great deal of literature on missing data imputation, but when and why would I ever use these methods when missing data for predictive (tree-based) models?
missing-data cart boosting data-imputation multiple-imputation
$endgroup$
add a comment |
$begingroup$
Everytime that I am making some predictive model and I have missing data I impute categorical variables with something like "UNKNOWN" and numerical variables with some absurd number that will never be seen in practice (even if the variable is unbounded I can take the exponent of the variable and make the unknown values negative).
The main advantage is that the model knows that the variable is missing, which is not the case for say mean imputation. I can see that this could be disastrous in linear models or neural networks but in tree-based models this is handled really smoothly.
I know that there is a great deal of literature on missing data imputation, but when and why would I ever use these methods when missing data for predictive (tree-based) models?
missing-data cart boosting data-imputation multiple-imputation
$endgroup$
$begingroup$
Imputing a large number for numeric data could be very bad for tree based models. Think of it this way, if your split is for example on income and the split is at say 100k, now everyone that was missing is going to be in the split with the high income earners
$endgroup$
– astel
8 hours ago
$begingroup$
The model will be fitted with that imputed values as well - so if they are significantly different than people with true high income the tree should make a split with true high and fake high (missing) income. If variability is low inside the tree node then there is not much to worry.
$endgroup$
– gsmafra
8 hours ago
add a comment |
$begingroup$
Everytime that I am making some predictive model and I have missing data I impute categorical variables with something like "UNKNOWN" and numerical variables with some absurd number that will never be seen in practice (even if the variable is unbounded I can take the exponent of the variable and make the unknown values negative).
The main advantage is that the model knows that the variable is missing, which is not the case for say mean imputation. I can see that this could be disastrous in linear models or neural networks but in tree-based models this is handled really smoothly.
I know that there is a great deal of literature on missing data imputation, but when and why would I ever use these methods when missing data for predictive (tree-based) models?
missing-data cart boosting data-imputation multiple-imputation
$endgroup$
Everytime that I am making some predictive model and I have missing data I impute categorical variables with something like "UNKNOWN" and numerical variables with some absurd number that will never be seen in practice (even if the variable is unbounded I can take the exponent of the variable and make the unknown values negative).
The main advantage is that the model knows that the variable is missing, which is not the case for say mean imputation. I can see that this could be disastrous in linear models or neural networks but in tree-based models this is handled really smoothly.
I know that there is a great deal of literature on missing data imputation, but when and why would I ever use these methods when missing data for predictive (tree-based) models?
missing-data cart boosting data-imputation multiple-imputation
missing-data cart boosting data-imputation multiple-imputation
asked 8 hours ago
gsmafragsmafra
17018
17018
$begingroup$
Imputing a large number for numeric data could be very bad for tree based models. Think of it this way, if your split is for example on income and the split is at say 100k, now everyone that was missing is going to be in the split with the high income earners
$endgroup$
– astel
8 hours ago
$begingroup$
The model will be fitted with that imputed values as well - so if they are significantly different than people with true high income the tree should make a split with true high and fake high (missing) income. If variability is low inside the tree node then there is not much to worry.
$endgroup$
– gsmafra
8 hours ago
add a comment |
$begingroup$
Imputing a large number for numeric data could be very bad for tree based models. Think of it this way, if your split is for example on income and the split is at say 100k, now everyone that was missing is going to be in the split with the high income earners
$endgroup$
– astel
8 hours ago
$begingroup$
The model will be fitted with that imputed values as well - so if they are significantly different than people with true high income the tree should make a split with true high and fake high (missing) income. If variability is low inside the tree node then there is not much to worry.
$endgroup$
– gsmafra
8 hours ago
$begingroup$
Imputing a large number for numeric data could be very bad for tree based models. Think of it this way, if your split is for example on income and the split is at say 100k, now everyone that was missing is going to be in the split with the high income earners
$endgroup$
– astel
8 hours ago
$begingroup$
Imputing a large number for numeric data could be very bad for tree based models. Think of it this way, if your split is for example on income and the split is at say 100k, now everyone that was missing is going to be in the split with the high income earners
$endgroup$
– astel
8 hours ago
$begingroup$
The model will be fitted with that imputed values as well - so if they are significantly different than people with true high income the tree should make a split with true high and fake high (missing) income. If variability is low inside the tree node then there is not much to worry.
$endgroup$
– gsmafra
8 hours ago
$begingroup$
The model will be fitted with that imputed values as well - so if they are significantly different than people with true high income the tree should make a split with true high and fake high (missing) income. If variability is low inside the tree node then there is not much to worry.
$endgroup$
– gsmafra
8 hours ago
add a comment |
2 Answers
2
active
oldest
votes
$begingroup$
One reason you may not want to use "insert impossible value" methods is that means that your predictive model works conditional on the distribution of the data missingness remaining unchanged. Thus, if after building your tree model, it is realized that we can start using certain features more often, we can no longer use the model that was built using the "impute impossible value" method without retraining the model.
In fact, this problem is even further compounded if the rates of missingness changes during the data collection process itself. Then, even immediately after building the model, it is already "out of date", as the current rates of missingness will be different than the rates of missingness during when the data was collected.
To illustrate the issue, let's suppose a bank is building a database to help predict if clients will default on a loan. Early in the data collection process, loan officers have the option to conduct a background investigation, but they almost never do for clients they deem as trustworthy. Thus, for the especially trustworthy customers, the background check variable is almost always missing. If you use the "impute impossible value" method, having a possible value for background checks indicates high risk.
If background check rates don't change at all, then this "impute impossible value" method will likely still provide valid predictions. However, let's suppose the bank realizes that background checks are really helpful for assessing risk, so they change their policy to include background checks for everyone. Then, everyone will have a possible value for background checks and using the "impute impossible value" method, everyone will be flagged as "high risk".
Cross validation will not catch this issue, as the missingness distribution will be the same between the training and testing sets. So even though the "impute impossible value" method may lead to pretty results during cross-validation, this will lead to poor predictions upon deployment!
Note that you will essentially need to throw away all your data everytime your data collection policy changes! Alternatively, if you can correctly impute the missing values and their uncertainty, you can now use the data that was collected under the old policy.
$endgroup$
$begingroup$
That's a good point, imputation could be more robust on changes in the way data is missing. I will take your statement on throwing away past data as an exaggeration though - including a time variable and retraining the model should be enough make it usable again.
$endgroup$
– gsmafra
7 hours ago
$begingroup$
@gsmafra: In general, I don't think adding a time variable will fix the problem. For example, in a random forest, the time variable will only be included in 1/3 of the trees, so it won't even be included in the majority of the decision trees in your random forest.
$endgroup$
– Cliff AB
6 hours ago
$begingroup$
To be clear, I don't think you should throw out your data...but I'd only advise doing "impossible value imputation" on variables you don't think will be very predictive to start with or you're fairly certain that the missingness distribution is fairly stable.
$endgroup$
– Cliff AB
6 hours ago
add a comment |
$begingroup$
I disagree. You can use "missing" as an additional set of information, just as anything else. Imputation, in this case, emphasizes the lack of information along some dimension in your sample. That could have additional explanatory power.
The issue is more that "missing" will become a dichotomic variable and mixing it with continous one can lead to inefficient state space segmentation in the way the regression trees are constructed. The key is to force the tree to only split dichotomically too, too increase efficiency. In other words, your split decision is already known beforehand.
It does not really matter whether the frequency of the missing values increase or decrease. This is more related to the problem being stationary or not and has nothing specifically to do with the decision tree.
Non-stationarity is handled via updating of decision trees or using an ensemble of decision trees.
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "65"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f397942%2fis-it-ever-recommended-to-use-mean-multiple-imputation-when-using-tree-based-pre%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
One reason you may not want to use "insert impossible value" methods is that means that your predictive model works conditional on the distribution of the data missingness remaining unchanged. Thus, if after building your tree model, it is realized that we can start using certain features more often, we can no longer use the model that was built using the "impute impossible value" method without retraining the model.
In fact, this problem is even further compounded if the rates of missingness changes during the data collection process itself. Then, even immediately after building the model, it is already "out of date", as the current rates of missingness will be different than the rates of missingness during when the data was collected.
To illustrate the issue, let's suppose a bank is building a database to help predict if clients will default on a loan. Early in the data collection process, loan officers have the option to conduct a background investigation, but they almost never do for clients they deem as trustworthy. Thus, for the especially trustworthy customers, the background check variable is almost always missing. If you use the "impute impossible value" method, having a possible value for background checks indicates high risk.
If background check rates don't change at all, then this "impute impossible value" method will likely still provide valid predictions. However, let's suppose the bank realizes that background checks are really helpful for assessing risk, so they change their policy to include background checks for everyone. Then, everyone will have a possible value for background checks and using the "impute impossible value" method, everyone will be flagged as "high risk".
Cross validation will not catch this issue, as the missingness distribution will be the same between the training and testing sets. So even though the "impute impossible value" method may lead to pretty results during cross-validation, this will lead to poor predictions upon deployment!
Note that you will essentially need to throw away all your data everytime your data collection policy changes! Alternatively, if you can correctly impute the missing values and their uncertainty, you can now use the data that was collected under the old policy.
$endgroup$
$begingroup$
That's a good point, imputation could be more robust on changes in the way data is missing. I will take your statement on throwing away past data as an exaggeration though - including a time variable and retraining the model should be enough make it usable again.
$endgroup$
– gsmafra
7 hours ago
$begingroup$
@gsmafra: In general, I don't think adding a time variable will fix the problem. For example, in a random forest, the time variable will only be included in 1/3 of the trees, so it won't even be included in the majority of the decision trees in your random forest.
$endgroup$
– Cliff AB
6 hours ago
$begingroup$
To be clear, I don't think you should throw out your data...but I'd only advise doing "impossible value imputation" on variables you don't think will be very predictive to start with or you're fairly certain that the missingness distribution is fairly stable.
$endgroup$
– Cliff AB
6 hours ago
add a comment |
$begingroup$
One reason you may not want to use "insert impossible value" methods is that means that your predictive model works conditional on the distribution of the data missingness remaining unchanged. Thus, if after building your tree model, it is realized that we can start using certain features more often, we can no longer use the model that was built using the "impute impossible value" method without retraining the model.
In fact, this problem is even further compounded if the rates of missingness changes during the data collection process itself. Then, even immediately after building the model, it is already "out of date", as the current rates of missingness will be different than the rates of missingness during when the data was collected.
To illustrate the issue, let's suppose a bank is building a database to help predict if clients will default on a loan. Early in the data collection process, loan officers have the option to conduct a background investigation, but they almost never do for clients they deem as trustworthy. Thus, for the especially trustworthy customers, the background check variable is almost always missing. If you use the "impute impossible value" method, having a possible value for background checks indicates high risk.
If background check rates don't change at all, then this "impute impossible value" method will likely still provide valid predictions. However, let's suppose the bank realizes that background checks are really helpful for assessing risk, so they change their policy to include background checks for everyone. Then, everyone will have a possible value for background checks and using the "impute impossible value" method, everyone will be flagged as "high risk".
Cross validation will not catch this issue, as the missingness distribution will be the same between the training and testing sets. So even though the "impute impossible value" method may lead to pretty results during cross-validation, this will lead to poor predictions upon deployment!
Note that you will essentially need to throw away all your data everytime your data collection policy changes! Alternatively, if you can correctly impute the missing values and their uncertainty, you can now use the data that was collected under the old policy.
$endgroup$
$begingroup$
That's a good point, imputation could be more robust on changes in the way data is missing. I will take your statement on throwing away past data as an exaggeration though - including a time variable and retraining the model should be enough make it usable again.
$endgroup$
– gsmafra
7 hours ago
$begingroup$
@gsmafra: In general, I don't think adding a time variable will fix the problem. For example, in a random forest, the time variable will only be included in 1/3 of the trees, so it won't even be included in the majority of the decision trees in your random forest.
$endgroup$
– Cliff AB
6 hours ago
$begingroup$
To be clear, I don't think you should throw out your data...but I'd only advise doing "impossible value imputation" on variables you don't think will be very predictive to start with or you're fairly certain that the missingness distribution is fairly stable.
$endgroup$
– Cliff AB
6 hours ago
add a comment |
$begingroup$
One reason you may not want to use "insert impossible value" methods is that means that your predictive model works conditional on the distribution of the data missingness remaining unchanged. Thus, if after building your tree model, it is realized that we can start using certain features more often, we can no longer use the model that was built using the "impute impossible value" method without retraining the model.
In fact, this problem is even further compounded if the rates of missingness changes during the data collection process itself. Then, even immediately after building the model, it is already "out of date", as the current rates of missingness will be different than the rates of missingness during when the data was collected.
To illustrate the issue, let's suppose a bank is building a database to help predict if clients will default on a loan. Early in the data collection process, loan officers have the option to conduct a background investigation, but they almost never do for clients they deem as trustworthy. Thus, for the especially trustworthy customers, the background check variable is almost always missing. If you use the "impute impossible value" method, having a possible value for background checks indicates high risk.
If background check rates don't change at all, then this "impute impossible value" method will likely still provide valid predictions. However, let's suppose the bank realizes that background checks are really helpful for assessing risk, so they change their policy to include background checks for everyone. Then, everyone will have a possible value for background checks and using the "impute impossible value" method, everyone will be flagged as "high risk".
Cross validation will not catch this issue, as the missingness distribution will be the same between the training and testing sets. So even though the "impute impossible value" method may lead to pretty results during cross-validation, this will lead to poor predictions upon deployment!
Note that you will essentially need to throw away all your data everytime your data collection policy changes! Alternatively, if you can correctly impute the missing values and their uncertainty, you can now use the data that was collected under the old policy.
$endgroup$
One reason you may not want to use "insert impossible value" methods is that means that your predictive model works conditional on the distribution of the data missingness remaining unchanged. Thus, if after building your tree model, it is realized that we can start using certain features more often, we can no longer use the model that was built using the "impute impossible value" method without retraining the model.
In fact, this problem is even further compounded if the rates of missingness changes during the data collection process itself. Then, even immediately after building the model, it is already "out of date", as the current rates of missingness will be different than the rates of missingness during when the data was collected.
To illustrate the issue, let's suppose a bank is building a database to help predict if clients will default on a loan. Early in the data collection process, loan officers have the option to conduct a background investigation, but they almost never do for clients they deem as trustworthy. Thus, for the especially trustworthy customers, the background check variable is almost always missing. If you use the "impute impossible value" method, having a possible value for background checks indicates high risk.
If background check rates don't change at all, then this "impute impossible value" method will likely still provide valid predictions. However, let's suppose the bank realizes that background checks are really helpful for assessing risk, so they change their policy to include background checks for everyone. Then, everyone will have a possible value for background checks and using the "impute impossible value" method, everyone will be flagged as "high risk".
Cross validation will not catch this issue, as the missingness distribution will be the same between the training and testing sets. So even though the "impute impossible value" method may lead to pretty results during cross-validation, this will lead to poor predictions upon deployment!
Note that you will essentially need to throw away all your data everytime your data collection policy changes! Alternatively, if you can correctly impute the missing values and their uncertainty, you can now use the data that was collected under the old policy.
edited 7 hours ago
answered 8 hours ago
Cliff ABCliff AB
13.5k12567
13.5k12567
$begingroup$
That's a good point, imputation could be more robust on changes in the way data is missing. I will take your statement on throwing away past data as an exaggeration though - including a time variable and retraining the model should be enough make it usable again.
$endgroup$
– gsmafra
7 hours ago
$begingroup$
@gsmafra: In general, I don't think adding a time variable will fix the problem. For example, in a random forest, the time variable will only be included in 1/3 of the trees, so it won't even be included in the majority of the decision trees in your random forest.
$endgroup$
– Cliff AB
6 hours ago
$begingroup$
To be clear, I don't think you should throw out your data...but I'd only advise doing "impossible value imputation" on variables you don't think will be very predictive to start with or you're fairly certain that the missingness distribution is fairly stable.
$endgroup$
– Cliff AB
6 hours ago
add a comment |
$begingroup$
That's a good point, imputation could be more robust on changes in the way data is missing. I will take your statement on throwing away past data as an exaggeration though - including a time variable and retraining the model should be enough make it usable again.
$endgroup$
– gsmafra
7 hours ago
$begingroup$
@gsmafra: In general, I don't think adding a time variable will fix the problem. For example, in a random forest, the time variable will only be included in 1/3 of the trees, so it won't even be included in the majority of the decision trees in your random forest.
$endgroup$
– Cliff AB
6 hours ago
$begingroup$
To be clear, I don't think you should throw out your data...but I'd only advise doing "impossible value imputation" on variables you don't think will be very predictive to start with or you're fairly certain that the missingness distribution is fairly stable.
$endgroup$
– Cliff AB
6 hours ago
$begingroup$
That's a good point, imputation could be more robust on changes in the way data is missing. I will take your statement on throwing away past data as an exaggeration though - including a time variable and retraining the model should be enough make it usable again.
$endgroup$
– gsmafra
7 hours ago
$begingroup$
That's a good point, imputation could be more robust on changes in the way data is missing. I will take your statement on throwing away past data as an exaggeration though - including a time variable and retraining the model should be enough make it usable again.
$endgroup$
– gsmafra
7 hours ago
$begingroup$
@gsmafra: In general, I don't think adding a time variable will fix the problem. For example, in a random forest, the time variable will only be included in 1/3 of the trees, so it won't even be included in the majority of the decision trees in your random forest.
$endgroup$
– Cliff AB
6 hours ago
$begingroup$
@gsmafra: In general, I don't think adding a time variable will fix the problem. For example, in a random forest, the time variable will only be included in 1/3 of the trees, so it won't even be included in the majority of the decision trees in your random forest.
$endgroup$
– Cliff AB
6 hours ago
$begingroup$
To be clear, I don't think you should throw out your data...but I'd only advise doing "impossible value imputation" on variables you don't think will be very predictive to start with or you're fairly certain that the missingness distribution is fairly stable.
$endgroup$
– Cliff AB
6 hours ago
$begingroup$
To be clear, I don't think you should throw out your data...but I'd only advise doing "impossible value imputation" on variables you don't think will be very predictive to start with or you're fairly certain that the missingness distribution is fairly stable.
$endgroup$
– Cliff AB
6 hours ago
add a comment |
$begingroup$
I disagree. You can use "missing" as an additional set of information, just as anything else. Imputation, in this case, emphasizes the lack of information along some dimension in your sample. That could have additional explanatory power.
The issue is more that "missing" will become a dichotomic variable and mixing it with continous one can lead to inefficient state space segmentation in the way the regression trees are constructed. The key is to force the tree to only split dichotomically too, too increase efficiency. In other words, your split decision is already known beforehand.
It does not really matter whether the frequency of the missing values increase or decrease. This is more related to the problem being stationary or not and has nothing specifically to do with the decision tree.
Non-stationarity is handled via updating of decision trees or using an ensemble of decision trees.
$endgroup$
add a comment |
$begingroup$
I disagree. You can use "missing" as an additional set of information, just as anything else. Imputation, in this case, emphasizes the lack of information along some dimension in your sample. That could have additional explanatory power.
The issue is more that "missing" will become a dichotomic variable and mixing it with continous one can lead to inefficient state space segmentation in the way the regression trees are constructed. The key is to force the tree to only split dichotomically too, too increase efficiency. In other words, your split decision is already known beforehand.
It does not really matter whether the frequency of the missing values increase or decrease. This is more related to the problem being stationary or not and has nothing specifically to do with the decision tree.
Non-stationarity is handled via updating of decision trees or using an ensemble of decision trees.
$endgroup$
add a comment |
$begingroup$
I disagree. You can use "missing" as an additional set of information, just as anything else. Imputation, in this case, emphasizes the lack of information along some dimension in your sample. That could have additional explanatory power.
The issue is more that "missing" will become a dichotomic variable and mixing it with continous one can lead to inefficient state space segmentation in the way the regression trees are constructed. The key is to force the tree to only split dichotomically too, too increase efficiency. In other words, your split decision is already known beforehand.
It does not really matter whether the frequency of the missing values increase or decrease. This is more related to the problem being stationary or not and has nothing specifically to do with the decision tree.
Non-stationarity is handled via updating of decision trees or using an ensemble of decision trees.
$endgroup$
I disagree. You can use "missing" as an additional set of information, just as anything else. Imputation, in this case, emphasizes the lack of information along some dimension in your sample. That could have additional explanatory power.
The issue is more that "missing" will become a dichotomic variable and mixing it with continous one can lead to inefficient state space segmentation in the way the regression trees are constructed. The key is to force the tree to only split dichotomically too, too increase efficiency. In other words, your split decision is already known beforehand.
It does not really matter whether the frequency of the missing values increase or decrease. This is more related to the problem being stationary or not and has nothing specifically to do with the decision tree.
Non-stationarity is handled via updating of decision trees or using an ensemble of decision trees.
answered 2 hours ago
Gkhan CebsGkhan Cebs
1543
1543
add a comment |
add a comment |
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f397942%2fis-it-ever-recommended-to-use-mean-multiple-imputation-when-using-tree-based-pre%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Imputing a large number for numeric data could be very bad for tree based models. Think of it this way, if your split is for example on income and the split is at say 100k, now everyone that was missing is going to be in the split with the high income earners
$endgroup$
– astel
8 hours ago
$begingroup$
The model will be fitted with that imputed values as well - so if they are significantly different than people with true high income the tree should make a split with true high and fake high (missing) income. If variability is low inside the tree node then there is not much to worry.
$endgroup$
– gsmafra
8 hours ago