In what order does sftp fetch files when using “get -r folder”?2019 Community Moderator ElectionHow to remove multiple files using sftpsftp command to get/download .tar.gz fileFile transfers with SSH and switch-userBacking up VPS via SSH/SFTPsftp - how to only copy files from folder that don't exist in destination folderWhy does default setfacl fail for nested directories?Disable Root Password Login… but can I SUDO with a GUI SFTP program?How to set sftp-server permissions and ownership in a shared sftp-folder? (ACL's ?)Getting error while using sftp (get: Invalid Flag -r)sftp get not working as expected
Why is "la Gestapo" feminine?
Isn't the word "experience" wrongly used in this context?
How do hiring committees for research positions view getting "scooped"?
How to remove space in section title at KOMA-Script
Inhabiting Mars versus going straight for a Dyson swarm
Do I need to convey a moral for each of my blog post?
Why didn’t Eve recognize the little cockroach as a living organism?
When did hardware antialiasing start being available?
What is the reasoning behind standardization (dividing by standard deviation)?
Knife as defense against stray dogs
Why is indicated airspeed rather than ground speed used during the takeoff roll?
How to test the sharpness of a knife?
How can I create URL shortcuts/redirects for task/diff IDs in Phabricator?
Is VPN a layer 3 concept?
TDE Master Key Rotation
Do people actually use the word "kaputt" in conversation?
is this saw blade faulty?
Is there any common country to visit for uk and schengen visa?
How to find the largest number(s) in a list of elements?
Is "inadequate referencing" a euphemism for plagiarism?
Friend wants my recommendation but I don't want to
Does Shadow Sorcerer's Eyes of the Dark work on all magical darkness or just his/hers?
How do you justify more code being written by following clean code practices?
Homology of the fiber
In what order does sftp fetch files when using “get -r folder”?
2019 Community Moderator ElectionHow to remove multiple files using sftpsftp command to get/download .tar.gz fileFile transfers with SSH and switch-userBacking up VPS via SSH/SFTPsftp - how to only copy files from folder that don't exist in destination folderWhy does default setfacl fail for nested directories?Disable Root Password Login… but can I SUDO with a GUI SFTP program?How to set sftp-server permissions and ownership in a shared sftp-folder? (ACL's ?)Getting error while using sftp (get: Invalid Flag -r)sftp get not working as expected
I'm doing a file transfer using sftp. Using the get -r folder command, I'm surprised about the order that the program is downloading the content.
It looks like it would be selecting the files it needs to download randomly. I can't believe that this is actually the case and I'm asking myself what's going on behind the scenes?
What's the order that sftp follows when downloading a folder with its content?
From what I can see so far, it is not by name nor by size.
file-copy sftp file-transfer
edited yesterday
Kusalananda
136k17257425
asked yesterday
AlexOnLinuxAlexOnLinux
1386
add a comment |
I'm doing a file transfer using sftp. Using the get -r folder command, I'm surprised about the order that the program is downloading the content.
It looks like it would be selecting the files it needs to download randomly. I can't believe that this is actually the case and I'm asking myself what's going on behind the scenes?
What's the order that sftp follows when downloading a folder with its content?
From what I can see so far, it is not by name nor by size.
file-copy sftp file-transfer
edited yesterday
Kusalananda
136k17257425
asked yesterday
AlexOnLinuxAlexOnLinux
1386
3
like would do a find command on a directory orls --sort=none: directory contents are not sorted
– A.B
yesterday
yes you are right, just checked it doing thels --sort=nonecomand.
– AlexOnLinux
yesterday
add a comment |
I'm doing a file transfer using sftp. Using the get -r folder command, I'm surprised about the order that the program is downloading the content.
It looks like it would be selecting the files it needs to download randomly. I can't believe that this is actually the case and I'm asking myself what's going on behind the scenes?
What's the order that sftp follows when downloading a folder with its content?
From what I can see so far, it is not by name nor by size.
file-copy sftp file-transfer
edited yesterday
Kusalananda
136k17257425
asked yesterday
AlexOnLinuxAlexOnLinux
1386
I'm doing a file transfer using sftp. Using the get -r folder command, I'm surprised about the order that the program is downloading the content.
It looks like it would be selecting the files it needs to download randomly. I can't believe that this is actually the case and I'm asking myself what's going on behind the scenes?
What's the order that sftp follows when downloading a folder with its content?
From what I can see so far, it is not by name nor by size.
file-copy sftp file-transfer
file-copy sftp file-transfer
edited yesterday
Kusalananda
136k17257425
asked yesterday
AlexOnLinuxAlexOnLinux
1386
edited yesterday
Kusalananda
136k17257425
asked yesterday
AlexOnLinuxAlexOnLinux
1386
edited yesterday
Kusalananda
136k17257425
edited yesterday
Kusalananda
136k17257425
edited yesterday
Kusalananda
136k17257425
136k17257425
asked yesterday
AlexOnLinuxAlexOnLinux
1386
asked yesterday
AlexOnLinuxAlexOnLinux
1386
asked yesterday
AlexOnLinuxAlexOnLinux
1386
1386
3
like would do a find command on a directory orls --sort=none: directory contents are not sorted
– A.B
yesterday
yes you are right, just checked it doing thels --sort=nonecomand.
– AlexOnLinux
yesterday
add a comment |
3
like would do a find command on a directory orls --sort=none: directory contents are not sorted
– A.B
yesterday
yes you are right, just checked it doing thels --sort=nonecomand.
– AlexOnLinux
yesterday
3
3
like would do a find command on a directory or
ls --sort=none: directory contents are not sorted– A.B
yesterday
like would do a find command on a directory or
ls --sort=none: directory contents are not sorted– A.B
yesterday
yes you are right, just checked it doing the
ls --sort=none comand.– AlexOnLinux
yesterday
yes you are right, just checked it doing the
ls --sort=none comand.– AlexOnLinux
yesterday
add a comment |
1 Answer
1
active
oldest
votes
When you list the directory contents with the ls command, it will sort the listing into alphanumeric order according to current locale's sorting rules by default. It is easy to assume that this is the "natural order" of things within the filesystem - but this isn't true.
Most filesystems don't sort their directories in any way: when adding a new file to a directory, the new file basically gets the first free slot in the directory's metadata structure. Sorting is only done when displaying the directory listing to the user. If a single directory has hundreds of thousands or millions of files in it, this sorting can actually require non-trivial amounts of memory and processing power.
When the order in which the files are processed does not matter, the most efficient way is to just read the directory metadata in order and process the files in the order encountered without any explicit sorting. In most cases this would mean the files will be processed basically in the order they were added to the directory, interspersed with newer files in cases where an old file was deleted and a later-added file reclaimed its metadata slot.
Some filesystems might use tree structures or something else in their internal design that might enforce a particular order for their directory entries as a side effect. But such an ordering might be based on inode numbers of the files or some other filesystem-internal detail, and so would not be guaranteed to be useful for humans for any practical purpose.
As @A.B said in the question comments, a find command or a ls -f or ls --sort=none would list the files without any explicit sorting, in whatever order the filesystem stores its directories.
edited yesterday
Kusalananda
136k17257425
answered yesterday
telcoMtelcoM
19.3k12448
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "106"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f506787%2fin-what-order-does-sftp-fetch-files-when-using-get-r-folder%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
When you list the directory contents with the ls command, it will sort the listing into alphanumeric order according to current locale's sorting rules by default. It is easy to assume that this is the "natural order" of things within the filesystem - but this isn't true.
Most filesystems don't sort their directories in any way: when adding a new file to a directory, the new file basically gets the first free slot in the directory's metadata structure. Sorting is only done when displaying the directory listing to the user. If a single directory has hundreds of thousands or millions of files in it, this sorting can actually require non-trivial amounts of memory and processing power.
When the order in which the files are processed does not matter, the most efficient way is to just read the directory metadata in order and process the files in the order encountered without any explicit sorting. In most cases this would mean the files will be processed basically in the order they were added to the directory, interspersed with newer files in cases where an old file was deleted and a later-added file reclaimed its metadata slot.
Some filesystems might use tree structures or something else in their internal design that might enforce a particular order for their directory entries as a side effect. But such an ordering might be based on inode numbers of the files or some other filesystem-internal detail, and so would not be guaranteed to be useful for humans for any practical purpose.
As @A.B said in the question comments, a find command or a ls -f or ls --sort=none would list the files without any explicit sorting, in whatever order the filesystem stores its directories.
edited yesterday
Kusalananda
136k17257425
answered yesterday
telcoMtelcoM
19.3k12448
add a comment |
When you list the directory contents with the ls command, it will sort the listing into alphanumeric order according to current locale's sorting rules by default. It is easy to assume that this is the "natural order" of things within the filesystem - but this isn't true.
Most filesystems don't sort their directories in any way: when adding a new file to a directory, the new file basically gets the first free slot in the directory's metadata structure. Sorting is only done when displaying the directory listing to the user. If a single directory has hundreds of thousands or millions of files in it, this sorting can actually require non-trivial amounts of memory and processing power.
When the order in which the files are processed does not matter, the most efficient way is to just read the directory metadata in order and process the files in the order encountered without any explicit sorting. In most cases this would mean the files will be processed basically in the order they were added to the directory, interspersed with newer files in cases where an old file was deleted and a later-added file reclaimed its metadata slot.
Some filesystems might use tree structures or something else in their internal design that might enforce a particular order for their directory entries as a side effect. But such an ordering might be based on inode numbers of the files or some other filesystem-internal detail, and so would not be guaranteed to be useful for humans for any practical purpose.
As @A.B said in the question comments, a find command or a ls -f or ls --sort=none would list the files without any explicit sorting, in whatever order the filesystem stores its directories.
edited yesterday
Kusalananda
136k17257425
answered yesterday
telcoMtelcoM
19.3k12448
add a comment |
When you list the directory contents with the ls command, it will sort the listing into alphanumeric order according to current locale's sorting rules by default. It is easy to assume that this is the "natural order" of things within the filesystem - but this isn't true.
Most filesystems don't sort their directories in any way: when adding a new file to a directory, the new file basically gets the first free slot in the directory's metadata structure. Sorting is only done when displaying the directory listing to the user. If a single directory has hundreds of thousands or millions of files in it, this sorting can actually require non-trivial amounts of memory and processing power.
When the order in which the files are processed does not matter, the most efficient way is to just read the directory metadata in order and process the files in the order encountered without any explicit sorting. In most cases this would mean the files will be processed basically in the order they were added to the directory, interspersed with newer files in cases where an old file was deleted and a later-added file reclaimed its metadata slot.
Some filesystems might use tree structures or something else in their internal design that might enforce a particular order for their directory entries as a side effect. But such an ordering might be based on inode numbers of the files or some other filesystem-internal detail, and so would not be guaranteed to be useful for humans for any practical purpose.
As @A.B said in the question comments, a find command or a ls -f or ls --sort=none would list the files without any explicit sorting, in whatever order the filesystem stores its directories.
edited yesterday
Kusalananda
136k17257425
answered yesterday
telcoMtelcoM
19.3k12448
When you list the directory contents with the ls command, it will sort the listing into alphanumeric order according to current locale's sorting rules by default. It is easy to assume that this is the "natural order" of things within the filesystem - but this isn't true.
Most filesystems don't sort their directories in any way: when adding a new file to a directory, the new file basically gets the first free slot in the directory's metadata structure. Sorting is only done when displaying the directory listing to the user. If a single directory has hundreds of thousands or millions of files in it, this sorting can actually require non-trivial amounts of memory and processing power.
When the order in which the files are processed does not matter, the most efficient way is to just read the directory metadata in order and process the files in the order encountered without any explicit sorting. In most cases this would mean the files will be processed basically in the order they were added to the directory, interspersed with newer files in cases where an old file was deleted and a later-added file reclaimed its metadata slot.
Some filesystems might use tree structures or something else in their internal design that might enforce a particular order for their directory entries as a side effect. But such an ordering might be based on inode numbers of the files or some other filesystem-internal detail, and so would not be guaranteed to be useful for humans for any practical purpose.
As @A.B said in the question comments, a find command or a ls -f or ls --sort=none would list the files without any explicit sorting, in whatever order the filesystem stores its directories.
edited yesterday
Kusalananda
136k17257425
answered yesterday
telcoMtelcoM
19.3k12448
edited yesterday
Kusalananda
136k17257425
edited yesterday
Kusalananda
136k17257425
edited yesterday
Kusalananda
136k17257425
136k17257425
answered yesterday
telcoMtelcoM
19.3k12448
answered yesterday
telcoMtelcoM
19.3k12448
answered yesterday
telcoMtelcoM
19.3k12448
19.3k12448
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f506787%2fin-what-order-does-sftp-fetch-files-when-using-get-r-folder%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
3
like would do a find command on a directory or
ls --sort=none: directory contents are not sorted– A.B
yesterday
yes you are right, just checked it doing the
ls --sort=nonecomand.– AlexOnLinux
yesterday