Does a log transform always bring a distribution closer to normal? Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)What is the reason the log transformation is used with right-skewed distributions?Confusion related to which transformation to useWhat distribution does this histogram look like?When should I perform transformation when analyzing skewed- data?How to log transform data with a large number of zerosTransforming extremely skewed distributionsDifference between log-normal distribution and logging variables, fitting normalOn log-normal distributionsHow to transform continuous data with extreme bimodal distributionTransforming a skewed data set to a Normal distributionHow to transform to normal distribution?log transform vs. resamplingTransform data used as response variable in mixed model to normal distributionLog of Ratio Results in Log-Normal Distribution?
Are my PIs rude or am I just being too sensitive?
How to recreate this effect in Photoshop?
ListPlot join points by nearest neighbor rather than order
When is phishing education going too far?
Should I call the interviewer directly, if HR aren't responding?
Is there a service that would inform me whenever a new direct route is scheduled from a given airport?
What is this single-engine low-wing propeller plane?
Why is "Captain Marvel" translated as male in Portugal?
When to stop saving and start investing?
Is there a Spanish version of "dot your i's and cross your t's" that includes the letter 'ñ'?
Disable hyphenation for an entire paragraph
What is the longest distance a 13th-level monk can jump while attacking on the same turn?
Is 1 ppb equal to 1 μg/kg?
Is it true to say that an hosting provider's DNS server is what links the entire hosting environment to ICANN?
Is a manifold-with-boundary with given interior and non-empty boundary essentially unique?
Why is black pepper both grey and black?
If Jon Snow became King of the Seven Kingdoms what would his regnal number be?
Gastric acid as a weapon
How does cp -a work
What is a Meta algorithm?
What are the motives behind Cersei's orders given to Bronn?
G-Code for resetting to 100% speed
How much radiation do nuclear physics experiments expose researchers to nowadays?
What do you call a plan that's an alternative plan in case your initial plan fails?
Does a log transform always bring a distribution closer to normal?
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)What is the reason the log transformation is used with right-skewed distributions?Confusion related to which transformation to useWhat distribution does this histogram look like?When should I perform transformation when analyzing skewed- data?How to log transform data with a large number of zerosTransforming extremely skewed distributionsDifference between log-normal distribution and logging variables, fitting normalOn log-normal distributionsHow to transform continuous data with extreme bimodal distributionTransforming a skewed data set to a Normal distributionHow to transform to normal distribution?log transform vs. resamplingTransform data used as response variable in mixed model to normal distributionLog of Ratio Results in Log-Normal Distribution?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
$begingroup$
I have a highly right skewed data set with a large range of values (from 1 ~ 10^6) (can't share the actual data for work related reasons).
When I plot the log of the data instead, the distribution looks a lot more like a normal distribution.
Have I stumbled on a meaningful insight in the data set, or is just a general property of the log transform that it brings the distribution closer to normal?
distributions normal-distribution data-transformation skewness
asked Mar 23 at 6:43
Akaike's ChildrenAkaike's Children
1457
$endgroup$
add a comment |
$begingroup$
I have a highly right skewed data set with a large range of values (from 1 ~ 10^6) (can't share the actual data for work related reasons).
When I plot the log of the data instead, the distribution looks a lot more like a normal distribution.
Have I stumbled on a meaningful insight in the data set, or is just a general property of the log transform that it brings the distribution closer to normal?
distributions normal-distribution data-transformation skewness
asked Mar 23 at 6:43
Akaike's ChildrenAkaike's Children
1457
$endgroup$
2
$begingroup$
I always naively assumed that the log transform works well if your data can be thought of as some constant, times many (more or less) independent factors close to 1. E.g. A guy's salary is 10% above the mean if he has a degree, 5% higher if he's living in a large town, 5% lower if he has health issues... A log transform turns that into a sum of independent small numbers, so you get a normal distribution.
$endgroup$
– nikie
Mar 23 at 12:49
$begingroup$
@Akaikes See here, here and particularly here & here which indicate that the log-transform won't always make even a right-skewed variate less skew (in absolute terms) than it was. A simple counterexample is the Maxwell(-Boltzmann) distribution, which is mildly right skew but the log of a Maxwell-variate is more strongly (left) skew.
$endgroup$
– Glen_b♦
Mar 24 at 2:20
add a comment |
$begingroup$
I have a highly right skewed data set with a large range of values (from 1 ~ 10^6) (can't share the actual data for work related reasons).
When I plot the log of the data instead, the distribution looks a lot more like a normal distribution.
Have I stumbled on a meaningful insight in the data set, or is just a general property of the log transform that it brings the distribution closer to normal?
distributions normal-distribution data-transformation skewness
asked Mar 23 at 6:43
Akaike's ChildrenAkaike's Children
1457
$endgroup$
I have a highly right skewed data set with a large range of values (from 1 ~ 10^6) (can't share the actual data for work related reasons).
When I plot the log of the data instead, the distribution looks a lot more like a normal distribution.
Have I stumbled on a meaningful insight in the data set, or is just a general property of the log transform that it brings the distribution closer to normal?
distributions normal-distribution data-transformation skewness
distributions normal-distribution data-transformation skewness
asked Mar 23 at 6:43
Akaike's ChildrenAkaike's Children
1457
asked Mar 23 at 6:43
Akaike's ChildrenAkaike's Children
1457
edited Mar 23 at 22:36
Akaike's Children
asked Mar 23 at 6:43
Akaike's ChildrenAkaike's Children
1457
asked Mar 23 at 6:43
Akaike's ChildrenAkaike's Children
1457
asked Mar 23 at 6:43
Akaike's ChildrenAkaike's Children
1457
1457
2
$begingroup$
I always naively assumed that the log transform works well if your data can be thought of as some constant, times many (more or less) independent factors close to 1. E.g. A guy's salary is 10% above the mean if he has a degree, 5% higher if he's living in a large town, 5% lower if he has health issues... A log transform turns that into a sum of independent small numbers, so you get a normal distribution.
$endgroup$
– nikie
Mar 23 at 12:49
$begingroup$
@Akaikes See here, here and particularly here & here which indicate that the log-transform won't always make even a right-skewed variate less skew (in absolute terms) than it was. A simple counterexample is the Maxwell(-Boltzmann) distribution, which is mildly right skew but the log of a Maxwell-variate is more strongly (left) skew.
$endgroup$
– Glen_b♦
Mar 24 at 2:20
add a comment |
2
$begingroup$
I always naively assumed that the log transform works well if your data can be thought of as some constant, times many (more or less) independent factors close to 1. E.g. A guy's salary is 10% above the mean if he has a degree, 5% higher if he's living in a large town, 5% lower if he has health issues... A log transform turns that into a sum of independent small numbers, so you get a normal distribution.
$endgroup$
– nikie
Mar 23 at 12:49
$begingroup$
@Akaikes See here, here and particularly here & here which indicate that the log-transform won't always make even a right-skewed variate less skew (in absolute terms) than it was. A simple counterexample is the Maxwell(-Boltzmann) distribution, which is mildly right skew but the log of a Maxwell-variate is more strongly (left) skew.
$endgroup$
– Glen_b♦
Mar 24 at 2:20
2
2
$begingroup$
I always naively assumed that the log transform works well if your data can be thought of as some constant, times many (more or less) independent factors close to 1. E.g. A guy's salary is 10% above the mean if he has a degree, 5% higher if he's living in a large town, 5% lower if he has health issues... A log transform turns that into a sum of independent small numbers, so you get a normal distribution.
$endgroup$
– nikie
Mar 23 at 12:49
$begingroup$
I always naively assumed that the log transform works well if your data can be thought of as some constant, times many (more or less) independent factors close to 1. E.g. A guy's salary is 10% above the mean if he has a degree, 5% higher if he's living in a large town, 5% lower if he has health issues... A log transform turns that into a sum of independent small numbers, so you get a normal distribution.
$endgroup$
– nikie
Mar 23 at 12:49
$begingroup$
@Akaikes See here, here and particularly here & here which indicate that the log-transform won't always make even a right-skewed variate less skew (in absolute terms) than it was. A simple counterexample is the Maxwell(-Boltzmann) distribution, which is mildly right skew but the log of a Maxwell-variate is more strongly (left) skew.
$endgroup$
– Glen_b♦
Mar 24 at 2:20
$begingroup$
@Akaikes See here, here and particularly here & here which indicate that the log-transform won't always make even a right-skewed variate less skew (in absolute terms) than it was. A simple counterexample is the Maxwell(-Boltzmann) distribution, which is mildly right skew but the log of a Maxwell-variate is more strongly (left) skew.
$endgroup$
– Glen_b♦
Mar 24 at 2:20
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
For purely positive quantities a log-transformation is indeed the standard first transformation to try and is very frequently used. It is also done if for regression you want a multiplicative interpretation of coefficients (e.g. doubling/ halving of blood cholesterol).
Of course it will not always make a distribution more normal, e.g. take samples from a N(1000, 1) distribution: any transformation can only make it less normal.
edited Mar 23 at 9:05
Nick Cox
39.4k588131
answered Mar 23 at 8:01
BjörnBjörn
11.7k11144
$endgroup$
4
$begingroup$
Similarly a distribution that is symmetric or left skewed will have its skewness made worse by logarithmic transformation. Consider the not very magnificent seven 1 2 3 4 5 6 7; then their square roots are left skewed and in the logarithms of those are even more left-skewed.
$endgroup$
– Nick Cox
Mar 23 at 9:04
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "65"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f398996%2fdoes-a-log-transform-always-bring-a-distribution-closer-to-normal%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
For purely positive quantities a log-transformation is indeed the standard first transformation to try and is very frequently used. It is also done if for regression you want a multiplicative interpretation of coefficients (e.g. doubling/ halving of blood cholesterol).
Of course it will not always make a distribution more normal, e.g. take samples from a N(1000, 1) distribution: any transformation can only make it less normal.
edited Mar 23 at 9:05
Nick Cox
39.4k588131
answered Mar 23 at 8:01
BjörnBjörn
11.7k11144
$endgroup$
4
$begingroup$
Similarly a distribution that is symmetric or left skewed will have its skewness made worse by logarithmic transformation. Consider the not very magnificent seven 1 2 3 4 5 6 7; then their square roots are left skewed and in the logarithms of those are even more left-skewed.
$endgroup$
– Nick Cox
Mar 23 at 9:04
add a comment |
$begingroup$
For purely positive quantities a log-transformation is indeed the standard first transformation to try and is very frequently used. It is also done if for regression you want a multiplicative interpretation of coefficients (e.g. doubling/ halving of blood cholesterol).
Of course it will not always make a distribution more normal, e.g. take samples from a N(1000, 1) distribution: any transformation can only make it less normal.
edited Mar 23 at 9:05
Nick Cox
39.4k588131
answered Mar 23 at 8:01
BjörnBjörn
11.7k11144
$endgroup$
4
$begingroup$
Similarly a distribution that is symmetric or left skewed will have its skewness made worse by logarithmic transformation. Consider the not very magnificent seven 1 2 3 4 5 6 7; then their square roots are left skewed and in the logarithms of those are even more left-skewed.
$endgroup$
– Nick Cox
Mar 23 at 9:04
add a comment |
$begingroup$
For purely positive quantities a log-transformation is indeed the standard first transformation to try and is very frequently used. It is also done if for regression you want a multiplicative interpretation of coefficients (e.g. doubling/ halving of blood cholesterol).
Of course it will not always make a distribution more normal, e.g. take samples from a N(1000, 1) distribution: any transformation can only make it less normal.
edited Mar 23 at 9:05
Nick Cox
39.4k588131
answered Mar 23 at 8:01
BjörnBjörn
11.7k11144
$endgroup$
For purely positive quantities a log-transformation is indeed the standard first transformation to try and is very frequently used. It is also done if for regression you want a multiplicative interpretation of coefficients (e.g. doubling/ halving of blood cholesterol).
Of course it will not always make a distribution more normal, e.g. take samples from a N(1000, 1) distribution: any transformation can only make it less normal.
edited Mar 23 at 9:05
Nick Cox
39.4k588131
answered Mar 23 at 8:01
BjörnBjörn
11.7k11144
edited Mar 23 at 9:05
Nick Cox
39.4k588131
edited Mar 23 at 9:05
Nick Cox
39.4k588131
edited Mar 23 at 9:05
Nick Cox
39.4k588131
39.4k588131
answered Mar 23 at 8:01
BjörnBjörn
11.7k11144
answered Mar 23 at 8:01
BjörnBjörn
11.7k11144
answered Mar 23 at 8:01
BjörnBjörn
11.7k11144
11.7k11144
4
$begingroup$
Similarly a distribution that is symmetric or left skewed will have its skewness made worse by logarithmic transformation. Consider the not very magnificent seven 1 2 3 4 5 6 7; then their square roots are left skewed and in the logarithms of those are even more left-skewed.
$endgroup$
– Nick Cox
Mar 23 at 9:04
add a comment |
4
$begingroup$
Similarly a distribution that is symmetric or left skewed will have its skewness made worse by logarithmic transformation. Consider the not very magnificent seven 1 2 3 4 5 6 7; then their square roots are left skewed and in the logarithms of those are even more left-skewed.
$endgroup$
– Nick Cox
Mar 23 at 9:04
4
4
$begingroup$
Similarly a distribution that is symmetric or left skewed will have its skewness made worse by logarithmic transformation. Consider the not very magnificent seven 1 2 3 4 5 6 7; then their square roots are left skewed and in the logarithms of those are even more left-skewed.
$endgroup$
– Nick Cox
Mar 23 at 9:04
$begingroup$
Similarly a distribution that is symmetric or left skewed will have its skewness made worse by logarithmic transformation. Consider the not very magnificent seven 1 2 3 4 5 6 7; then their square roots are left skewed and in the logarithms of those are even more left-skewed.
$endgroup$
– Nick Cox
Mar 23 at 9:04
add a comment |
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f398996%2fdoes-a-log-transform-always-bring-a-distribution-closer-to-normal%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
$begingroup$
I always naively assumed that the log transform works well if your data can be thought of as some constant, times many (more or less) independent factors close to 1. E.g. A guy's salary is 10% above the mean if he has a degree, 5% higher if he's living in a large town, 5% lower if he has health issues... A log transform turns that into a sum of independent small numbers, so you get a normal distribution.
$endgroup$
– nikie
Mar 23 at 12:49
$begingroup$
@Akaikes See here, here and particularly here & here which indicate that the log-transform won't always make even a right-skewed variate less skew (in absolute terms) than it was. A simple counterexample is the Maxwell(-Boltzmann) distribution, which is mildly right skew but the log of a Maxwell-variate is more strongly (left) skew.
$endgroup$
– Glen_b♦
Mar 24 at 2:20