Why is .bash_history periodically wiped?
This is the second time it's happened.
I just tried grep-ing some historical commands and came up empty. a look at my bash history shows that it's practically empty. Like it's been wiped clean.
I use iTerm2 and on MacOS 10.13.6.
I see there's a .bash_sessions directory with some sessions saved but I guess this is probably an iTerm2 thing to preserve sessions for some reason or another.
terminal bash iterm
add a comment |
This is the second time it's happened.
I just tried grep-ing some historical commands and came up empty. a look at my bash history shows that it's practically empty. Like it's been wiped clean.
I use iTerm2 and on MacOS 10.13.6.
I see there's a .bash_sessions directory with some sessions saved but I guess this is probably an iTerm2 thing to preserve sessions for some reason or another.
terminal bash iterm
unix.stackexchange.com/questions/163371/…
– JBis
yesterday
add a comment |
This is the second time it's happened.
I just tried grep-ing some historical commands and came up empty. a look at my bash history shows that it's practically empty. Like it's been wiped clean.
I use iTerm2 and on MacOS 10.13.6.
I see there's a .bash_sessions directory with some sessions saved but I guess this is probably an iTerm2 thing to preserve sessions for some reason or another.
terminal bash iterm
This is the second time it's happened.
I just tried grep-ing some historical commands and came up empty. a look at my bash history shows that it's practically empty. Like it's been wiped clean.
I use iTerm2 and on MacOS 10.13.6.
I see there's a .bash_sessions directory with some sessions saved but I guess this is probably an iTerm2 thing to preserve sessions for some reason or another.
terminal bash iterm
terminal bash iterm
asked yesterday
Dark Star1Dark Star1
249211
249211
unix.stackexchange.com/questions/163371/…
– JBis
yesterday
add a comment |
unix.stackexchange.com/questions/163371/…
– JBis
yesterday
unix.stackexchange.com/questions/163371/…
– JBis
yesterday
unix.stackexchange.com/questions/163371/…
– JBis
yesterday
add a comment |
1 Answer
1
active
oldest
votes
There can be multiple reasons as to why this happens - I'll try to outline how it works below.
However, I can say that the other answer you have received here is not correct. HISTFILESIZE and HISTSIZE will not cause your history to "sometimes" being entirely wiped out or almost wiped out. Only by setting them to 0 you would get nothing in the files - but it would happen every time, and not by chance. In addition, what you describe with the file being "almost wiped out" cannot happen due to it being set to 0.
You're actually on to the right thing yourself by mentioning .bash_sessions. That is not an iTerm2 "thing", but rather how it works by default on a standard macOS install. Apple has built-in per-session history on top of a regular bash install.
This means that if you have multiple terminals running (for example multiple tabs), each of those will have a seperate history tracked in .bash_sessions. If you reboot your Mac and the terminal windows are restored, you'll find that each still have their own history - and only their own history.
When you close down a bash session, Apple's system will merge the history for that specific session into the global .bash_history file. Then when you open a new terminal (and thus bash session), it will start with that merged history containing history from potentially multiple sessions.
This is all handled by the /etc/bashrc_Apple_Terminal
script.
Now that you know how it works, here are some possible causes for what you're seeing:
You may accidentially have removed the history yourself (history -c)
Your Mac could be infected with malware and/or hacked, and someone else is removing your .bash_history file
You have software installed that periodically empties/removes .bash_history
These reasons are not that likely, I would say.
Now there's one last possible cause:
There's a known race-condition in handling the bash histories. If you close down multiple bash sessions at once (for example closing down a terminal window with multiple tabs), you might loose part of or the whole history.
This happens if the computer is "slow" while reading/writing parts of the history. Basically how it works is that your currently history is read, the file is deleted/moved away, and a new file is created, where all the history lines are written to. If this process is running multiple times at the same time, you might see that one process removes the old history, the other process read an empty history, then the first process writes out the whole history again, and then the other process deletes that and writes out a very short history containing only few recent lines - as it didn't read anything in. This bug shows up as exactly what you have described.
The fix is simply to close one bash session at a time to let the system have time to write out everything correctly.
Thanks for your explanation I will see if I can replicate this given your answer and get back to you. I am aware that it may not necessarily happen on the first try, but I'd like to be sure it is replicable.
– Dark Star1
yesterday
Also just want to add to the malware part - a common OS X RAT known as Eggshell has a macOS payload that actually executeshistory -wc
as part of its code after the main payload, such that all traces of the payload injection command are wiped. Not saying its necessary Eggshell but it could be something common if someone wanted to hide their traces...
– QuickishFM
yesterday
This has only happened to me twice and I doubt it is that malware but thanks for the info. Ill look out for it just in case
– Dark Star1
yesterday
As a workaround, I have this in my bashrc:export PROMPT_COMMAND='echo $(date +%Y-%m-%dT%H:%M:%S) $(pwd): "$(history 1 | head -c 512 | sed "s/^[ ]*[0-9]+[ ]*//" )" >> ~/.bash_history_full; echo -ne "33]0;${PWD##*/}07"'
which will store each command after it is executed.
– jpa
20 hours ago
The disadvantage there is thought that it won't seperate sessions for you - so it will be hard to make sense of stuff afterwards if you have many sessions (for example some local and some remote).
– jksoegaard
19 hours ago
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "118"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fapple.stackexchange.com%2fquestions%2f354746%2fwhy-is-bash-history-periodically-wiped%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
There can be multiple reasons as to why this happens - I'll try to outline how it works below.
However, I can say that the other answer you have received here is not correct. HISTFILESIZE and HISTSIZE will not cause your history to "sometimes" being entirely wiped out or almost wiped out. Only by setting them to 0 you would get nothing in the files - but it would happen every time, and not by chance. In addition, what you describe with the file being "almost wiped out" cannot happen due to it being set to 0.
You're actually on to the right thing yourself by mentioning .bash_sessions. That is not an iTerm2 "thing", but rather how it works by default on a standard macOS install. Apple has built-in per-session history on top of a regular bash install.
This means that if you have multiple terminals running (for example multiple tabs), each of those will have a seperate history tracked in .bash_sessions. If you reboot your Mac and the terminal windows are restored, you'll find that each still have their own history - and only their own history.
When you close down a bash session, Apple's system will merge the history for that specific session into the global .bash_history file. Then when you open a new terminal (and thus bash session), it will start with that merged history containing history from potentially multiple sessions.
This is all handled by the /etc/bashrc_Apple_Terminal
script.
Now that you know how it works, here are some possible causes for what you're seeing:
You may accidentially have removed the history yourself (history -c)
Your Mac could be infected with malware and/or hacked, and someone else is removing your .bash_history file
You have software installed that periodically empties/removes .bash_history
These reasons are not that likely, I would say.
Now there's one last possible cause:
There's a known race-condition in handling the bash histories. If you close down multiple bash sessions at once (for example closing down a terminal window with multiple tabs), you might loose part of or the whole history.
This happens if the computer is "slow" while reading/writing parts of the history. Basically how it works is that your currently history is read, the file is deleted/moved away, and a new file is created, where all the history lines are written to. If this process is running multiple times at the same time, you might see that one process removes the old history, the other process read an empty history, then the first process writes out the whole history again, and then the other process deletes that and writes out a very short history containing only few recent lines - as it didn't read anything in. This bug shows up as exactly what you have described.
The fix is simply to close one bash session at a time to let the system have time to write out everything correctly.
Thanks for your explanation I will see if I can replicate this given your answer and get back to you. I am aware that it may not necessarily happen on the first try, but I'd like to be sure it is replicable.
– Dark Star1
yesterday
Also just want to add to the malware part - a common OS X RAT known as Eggshell has a macOS payload that actually executeshistory -wc
as part of its code after the main payload, such that all traces of the payload injection command are wiped. Not saying its necessary Eggshell but it could be something common if someone wanted to hide their traces...
– QuickishFM
yesterday
This has only happened to me twice and I doubt it is that malware but thanks for the info. Ill look out for it just in case
– Dark Star1
yesterday
As a workaround, I have this in my bashrc:export PROMPT_COMMAND='echo $(date +%Y-%m-%dT%H:%M:%S) $(pwd): "$(history 1 | head -c 512 | sed "s/^[ ]*[0-9]+[ ]*//" )" >> ~/.bash_history_full; echo -ne "33]0;${PWD##*/}07"'
which will store each command after it is executed.
– jpa
20 hours ago
The disadvantage there is thought that it won't seperate sessions for you - so it will be hard to make sense of stuff afterwards if you have many sessions (for example some local and some remote).
– jksoegaard
19 hours ago
add a comment |
There can be multiple reasons as to why this happens - I'll try to outline how it works below.
However, I can say that the other answer you have received here is not correct. HISTFILESIZE and HISTSIZE will not cause your history to "sometimes" being entirely wiped out or almost wiped out. Only by setting them to 0 you would get nothing in the files - but it would happen every time, and not by chance. In addition, what you describe with the file being "almost wiped out" cannot happen due to it being set to 0.
You're actually on to the right thing yourself by mentioning .bash_sessions. That is not an iTerm2 "thing", but rather how it works by default on a standard macOS install. Apple has built-in per-session history on top of a regular bash install.
This means that if you have multiple terminals running (for example multiple tabs), each of those will have a seperate history tracked in .bash_sessions. If you reboot your Mac and the terminal windows are restored, you'll find that each still have their own history - and only their own history.
When you close down a bash session, Apple's system will merge the history for that specific session into the global .bash_history file. Then when you open a new terminal (and thus bash session), it will start with that merged history containing history from potentially multiple sessions.
This is all handled by the /etc/bashrc_Apple_Terminal
script.
Now that you know how it works, here are some possible causes for what you're seeing:
You may accidentially have removed the history yourself (history -c)
Your Mac could be infected with malware and/or hacked, and someone else is removing your .bash_history file
You have software installed that periodically empties/removes .bash_history
These reasons are not that likely, I would say.
Now there's one last possible cause:
There's a known race-condition in handling the bash histories. If you close down multiple bash sessions at once (for example closing down a terminal window with multiple tabs), you might loose part of or the whole history.
This happens if the computer is "slow" while reading/writing parts of the history. Basically how it works is that your currently history is read, the file is deleted/moved away, and a new file is created, where all the history lines are written to. If this process is running multiple times at the same time, you might see that one process removes the old history, the other process read an empty history, then the first process writes out the whole history again, and then the other process deletes that and writes out a very short history containing only few recent lines - as it didn't read anything in. This bug shows up as exactly what you have described.
The fix is simply to close one bash session at a time to let the system have time to write out everything correctly.
Thanks for your explanation I will see if I can replicate this given your answer and get back to you. I am aware that it may not necessarily happen on the first try, but I'd like to be sure it is replicable.
– Dark Star1
yesterday
Also just want to add to the malware part - a common OS X RAT known as Eggshell has a macOS payload that actually executeshistory -wc
as part of its code after the main payload, such that all traces of the payload injection command are wiped. Not saying its necessary Eggshell but it could be something common if someone wanted to hide their traces...
– QuickishFM
yesterday
This has only happened to me twice and I doubt it is that malware but thanks for the info. Ill look out for it just in case
– Dark Star1
yesterday
As a workaround, I have this in my bashrc:export PROMPT_COMMAND='echo $(date +%Y-%m-%dT%H:%M:%S) $(pwd): "$(history 1 | head -c 512 | sed "s/^[ ]*[0-9]+[ ]*//" )" >> ~/.bash_history_full; echo -ne "33]0;${PWD##*/}07"'
which will store each command after it is executed.
– jpa
20 hours ago
The disadvantage there is thought that it won't seperate sessions for you - so it will be hard to make sense of stuff afterwards if you have many sessions (for example some local and some remote).
– jksoegaard
19 hours ago
add a comment |
There can be multiple reasons as to why this happens - I'll try to outline how it works below.
However, I can say that the other answer you have received here is not correct. HISTFILESIZE and HISTSIZE will not cause your history to "sometimes" being entirely wiped out or almost wiped out. Only by setting them to 0 you would get nothing in the files - but it would happen every time, and not by chance. In addition, what you describe with the file being "almost wiped out" cannot happen due to it being set to 0.
You're actually on to the right thing yourself by mentioning .bash_sessions. That is not an iTerm2 "thing", but rather how it works by default on a standard macOS install. Apple has built-in per-session history on top of a regular bash install.
This means that if you have multiple terminals running (for example multiple tabs), each of those will have a seperate history tracked in .bash_sessions. If you reboot your Mac and the terminal windows are restored, you'll find that each still have their own history - and only their own history.
When you close down a bash session, Apple's system will merge the history for that specific session into the global .bash_history file. Then when you open a new terminal (and thus bash session), it will start with that merged history containing history from potentially multiple sessions.
This is all handled by the /etc/bashrc_Apple_Terminal
script.
Now that you know how it works, here are some possible causes for what you're seeing:
You may accidentially have removed the history yourself (history -c)
Your Mac could be infected with malware and/or hacked, and someone else is removing your .bash_history file
You have software installed that periodically empties/removes .bash_history
These reasons are not that likely, I would say.
Now there's one last possible cause:
There's a known race-condition in handling the bash histories. If you close down multiple bash sessions at once (for example closing down a terminal window with multiple tabs), you might loose part of or the whole history.
This happens if the computer is "slow" while reading/writing parts of the history. Basically how it works is that your currently history is read, the file is deleted/moved away, and a new file is created, where all the history lines are written to. If this process is running multiple times at the same time, you might see that one process removes the old history, the other process read an empty history, then the first process writes out the whole history again, and then the other process deletes that and writes out a very short history containing only few recent lines - as it didn't read anything in. This bug shows up as exactly what you have described.
The fix is simply to close one bash session at a time to let the system have time to write out everything correctly.
There can be multiple reasons as to why this happens - I'll try to outline how it works below.
However, I can say that the other answer you have received here is not correct. HISTFILESIZE and HISTSIZE will not cause your history to "sometimes" being entirely wiped out or almost wiped out. Only by setting them to 0 you would get nothing in the files - but it would happen every time, and not by chance. In addition, what you describe with the file being "almost wiped out" cannot happen due to it being set to 0.
You're actually on to the right thing yourself by mentioning .bash_sessions. That is not an iTerm2 "thing", but rather how it works by default on a standard macOS install. Apple has built-in per-session history on top of a regular bash install.
This means that if you have multiple terminals running (for example multiple tabs), each of those will have a seperate history tracked in .bash_sessions. If you reboot your Mac and the terminal windows are restored, you'll find that each still have their own history - and only their own history.
When you close down a bash session, Apple's system will merge the history for that specific session into the global .bash_history file. Then when you open a new terminal (and thus bash session), it will start with that merged history containing history from potentially multiple sessions.
This is all handled by the /etc/bashrc_Apple_Terminal
script.
Now that you know how it works, here are some possible causes for what you're seeing:
You may accidentially have removed the history yourself (history -c)
Your Mac could be infected with malware and/or hacked, and someone else is removing your .bash_history file
You have software installed that periodically empties/removes .bash_history
These reasons are not that likely, I would say.
Now there's one last possible cause:
There's a known race-condition in handling the bash histories. If you close down multiple bash sessions at once (for example closing down a terminal window with multiple tabs), you might loose part of or the whole history.
This happens if the computer is "slow" while reading/writing parts of the history. Basically how it works is that your currently history is read, the file is deleted/moved away, and a new file is created, where all the history lines are written to. If this process is running multiple times at the same time, you might see that one process removes the old history, the other process read an empty history, then the first process writes out the whole history again, and then the other process deletes that and writes out a very short history containing only few recent lines - as it didn't read anything in. This bug shows up as exactly what you have described.
The fix is simply to close one bash session at a time to let the system have time to write out everything correctly.
answered yesterday
jksoegaardjksoegaard
19.6k2150
19.6k2150
Thanks for your explanation I will see if I can replicate this given your answer and get back to you. I am aware that it may not necessarily happen on the first try, but I'd like to be sure it is replicable.
– Dark Star1
yesterday
Also just want to add to the malware part - a common OS X RAT known as Eggshell has a macOS payload that actually executeshistory -wc
as part of its code after the main payload, such that all traces of the payload injection command are wiped. Not saying its necessary Eggshell but it could be something common if someone wanted to hide their traces...
– QuickishFM
yesterday
This has only happened to me twice and I doubt it is that malware but thanks for the info. Ill look out for it just in case
– Dark Star1
yesterday
As a workaround, I have this in my bashrc:export PROMPT_COMMAND='echo $(date +%Y-%m-%dT%H:%M:%S) $(pwd): "$(history 1 | head -c 512 | sed "s/^[ ]*[0-9]+[ ]*//" )" >> ~/.bash_history_full; echo -ne "33]0;${PWD##*/}07"'
which will store each command after it is executed.
– jpa
20 hours ago
The disadvantage there is thought that it won't seperate sessions for you - so it will be hard to make sense of stuff afterwards if you have many sessions (for example some local and some remote).
– jksoegaard
19 hours ago
add a comment |
Thanks for your explanation I will see if I can replicate this given your answer and get back to you. I am aware that it may not necessarily happen on the first try, but I'd like to be sure it is replicable.
– Dark Star1
yesterday
Also just want to add to the malware part - a common OS X RAT known as Eggshell has a macOS payload that actually executeshistory -wc
as part of its code after the main payload, such that all traces of the payload injection command are wiped. Not saying its necessary Eggshell but it could be something common if someone wanted to hide their traces...
– QuickishFM
yesterday
This has only happened to me twice and I doubt it is that malware but thanks for the info. Ill look out for it just in case
– Dark Star1
yesterday
As a workaround, I have this in my bashrc:export PROMPT_COMMAND='echo $(date +%Y-%m-%dT%H:%M:%S) $(pwd): "$(history 1 | head -c 512 | sed "s/^[ ]*[0-9]+[ ]*//" )" >> ~/.bash_history_full; echo -ne "33]0;${PWD##*/}07"'
which will store each command after it is executed.
– jpa
20 hours ago
The disadvantage there is thought that it won't seperate sessions for you - so it will be hard to make sense of stuff afterwards if you have many sessions (for example some local and some remote).
– jksoegaard
19 hours ago
Thanks for your explanation I will see if I can replicate this given your answer and get back to you. I am aware that it may not necessarily happen on the first try, but I'd like to be sure it is replicable.
– Dark Star1
yesterday
Thanks for your explanation I will see if I can replicate this given your answer and get back to you. I am aware that it may not necessarily happen on the first try, but I'd like to be sure it is replicable.
– Dark Star1
yesterday
Also just want to add to the malware part - a common OS X RAT known as Eggshell has a macOS payload that actually executes
history -wc
as part of its code after the main payload, such that all traces of the payload injection command are wiped. Not saying its necessary Eggshell but it could be something common if someone wanted to hide their traces...– QuickishFM
yesterday
Also just want to add to the malware part - a common OS X RAT known as Eggshell has a macOS payload that actually executes
history -wc
as part of its code after the main payload, such that all traces of the payload injection command are wiped. Not saying its necessary Eggshell but it could be something common if someone wanted to hide their traces...– QuickishFM
yesterday
This has only happened to me twice and I doubt it is that malware but thanks for the info. Ill look out for it just in case
– Dark Star1
yesterday
This has only happened to me twice and I doubt it is that malware but thanks for the info. Ill look out for it just in case
– Dark Star1
yesterday
As a workaround, I have this in my bashrc:
export PROMPT_COMMAND='echo $(date +%Y-%m-%dT%H:%M:%S) $(pwd): "$(history 1 | head -c 512 | sed "s/^[ ]*[0-9]+[ ]*//" )" >> ~/.bash_history_full; echo -ne "33]0;${PWD##*/}07"'
which will store each command after it is executed.– jpa
20 hours ago
As a workaround, I have this in my bashrc:
export PROMPT_COMMAND='echo $(date +%Y-%m-%dT%H:%M:%S) $(pwd): "$(history 1 | head -c 512 | sed "s/^[ ]*[0-9]+[ ]*//" )" >> ~/.bash_history_full; echo -ne "33]0;${PWD##*/}07"'
which will store each command after it is executed.– jpa
20 hours ago
The disadvantage there is thought that it won't seperate sessions for you - so it will be hard to make sense of stuff afterwards if you have many sessions (for example some local and some remote).
– jksoegaard
19 hours ago
The disadvantage there is thought that it won't seperate sessions for you - so it will be hard to make sense of stuff afterwards if you have many sessions (for example some local and some remote).
– jksoegaard
19 hours ago
add a comment |
Thanks for contributing an answer to Ask Different!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fapple.stackexchange.com%2fquestions%2f354746%2fwhy-is-bash-history-periodically-wiped%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
unix.stackexchange.com/questions/163371/…
– JBis
yesterday