Bash - pair each line of file2019 Community Moderator ElectionCommand line tool to “cat” pairwise expansion of all rows in a fileCost-efficiently pair each line of a file with all othersHow to rewrite multiline path into one-line relative pathMerge two files line by line with the delimiter triple pipe symbol “|||”How do I remove all but the file name (with no extension) from a full file path?Insert string or line after last instance of a specific search variable, in a loopcopy every line from a text file that contains a number greater than 5000How to insert a line from file A above the FIRST LINE in file Bextract lines from a file based on sequential pair of patterns, and output to separate filesscript to parse file for two consecutive lines of unequal lengthHow to add a path before the first character of each line in a file.txt and save the same file with those edits?How can I add 10 lines from a file (file2) to another one after 2 lines (file1)?

Using an older 200A breaker panel on a 60A feeder circuit from house?

Taking the numerator and the denominator

Offset in split text content

How to test the sharpness of a knife?

Recursively move files within sub directories

categorizing a variable turns it from insignificant to significant

Rendered textures different to 3D View

Should a narrator ever describe things based on a character's view instead of facts?

I keep switching characters, how do I stop?

Why didn’t Eve recognize the little cockroach as a living organism?

Put the phone down / Put down the phone

Are hand made posters acceptable in Academia?

Why does the frost depth increase when the surface temperature warms up?

Checking @@ROWCOUNT failing

What is the period/term used describe Giuseppe Arcimboldo's style of painting?

What is this high flying aircraft over Pennsylvania?

1 John in Luther’s Bibel

Amorphous proper classes in MK

What is it called when someone votes for an option that's not their first choice?

Magnifying glass in hyperbolic space

How can I, as DM, avoid the Conga Line of Death occurring when implementing some form of flanking rule?

How would a solely written language work mechanically

Has the laser at Magurele, Romania reached a tenth of the Sun's power?

How do you say "Trust your struggle." in French?



Bash - pair each line of file



2019 Community Moderator ElectionCommand line tool to “cat” pairwise expansion of all rows in a fileCost-efficiently pair each line of a file with all othersHow to rewrite multiline path into one-line relative pathMerge two files line by line with the delimiter triple pipe symbol “|||”How do I remove all but the file name (with no extension) from a full file path?Insert string or line after last instance of a specific search variable, in a loopcopy every line from a text file that contains a number greater than 5000How to insert a line from file A above the FIRST LINE in file Bextract lines from a file based on sequential pair of patterns, and output to separate filesscript to parse file for two consecutive lines of unequal lengthHow to add a path before the first character of each line in a file.txt and save the same file with those edits?How can I add 10 lines from a file (file2) to another one after 2 lines (file1)?










9















This question is strongly related to this and this question. I have a file that contains several lines where each line is a path to a file. Now I want to pair each line with each different line (not itself). Also a pair A B is equal to a B A pair for my purposes, so only one of these combinations should be produced.



Example



files.dat reads like this in a shorthand notation, each letter is a file path (absolute or relative)



a
b
c
d
e


Then my result should look something like this:



a b
a c
a d
a e
b c
b d
b e
c d
c e
d e


Preferrably I would like to solve this in bash. Unlike the other questions, my file list is rather small (about 200 lines), so using loops and RAM capacity
pose no problems.










share|improve this question









New contributor




Enno is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.




















  • Does it have to be in bash proper, or just something available via the bash commandline? Other utilities are better positioned to process text.

    – Jeff Schaller
    2 days ago











  • @JeffSchaller Something accessible via the bash commandline. I was a bit unclear, sorry

    – Enno
    2 days ago












  • This is almost becoming a Code Golf :P

    – Richard de Wit
    yesterday






  • 1





    As a general rule, as long as you need to do something non-trivial, use your favourite scripting language over BASH. It will be less fragile (for example, against special characters or spaces), and much easier to expand whenever you need it (if you need three, or filter some of them away). Python or Perl should be installed in almost any Linux box, so they are good choices (unless you are working on embedded systems, like Busybox).

    – Davidmh
    yesterday















9















This question is strongly related to this and this question. I have a file that contains several lines where each line is a path to a file. Now I want to pair each line with each different line (not itself). Also a pair A B is equal to a B A pair for my purposes, so only one of these combinations should be produced.



Example



files.dat reads like this in a shorthand notation, each letter is a file path (absolute or relative)



a
b
c
d
e


Then my result should look something like this:



a b
a c
a d
a e
b c
b d
b e
c d
c e
d e


Preferrably I would like to solve this in bash. Unlike the other questions, my file list is rather small (about 200 lines), so using loops and RAM capacity
pose no problems.










share|improve this question









New contributor




Enno is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.




















  • Does it have to be in bash proper, or just something available via the bash commandline? Other utilities are better positioned to process text.

    – Jeff Schaller
    2 days ago











  • @JeffSchaller Something accessible via the bash commandline. I was a bit unclear, sorry

    – Enno
    2 days ago












  • This is almost becoming a Code Golf :P

    – Richard de Wit
    yesterday






  • 1





    As a general rule, as long as you need to do something non-trivial, use your favourite scripting language over BASH. It will be less fragile (for example, against special characters or spaces), and much easier to expand whenever you need it (if you need three, or filter some of them away). Python or Perl should be installed in almost any Linux box, so they are good choices (unless you are working on embedded systems, like Busybox).

    – Davidmh
    yesterday













9












9








9


0






This question is strongly related to this and this question. I have a file that contains several lines where each line is a path to a file. Now I want to pair each line with each different line (not itself). Also a pair A B is equal to a B A pair for my purposes, so only one of these combinations should be produced.



Example



files.dat reads like this in a shorthand notation, each letter is a file path (absolute or relative)



a
b
c
d
e


Then my result should look something like this:



a b
a c
a d
a e
b c
b d
b e
c d
c e
d e


Preferrably I would like to solve this in bash. Unlike the other questions, my file list is rather small (about 200 lines), so using loops and RAM capacity
pose no problems.










share|improve this question









New contributor




Enno is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.












This question is strongly related to this and this question. I have a file that contains several lines where each line is a path to a file. Now I want to pair each line with each different line (not itself). Also a pair A B is equal to a B A pair for my purposes, so only one of these combinations should be produced.



Example



files.dat reads like this in a shorthand notation, each letter is a file path (absolute or relative)



a
b
c
d
e


Then my result should look something like this:



a b
a c
a d
a e
b c
b d
b e
c d
c e
d e


Preferrably I would like to solve this in bash. Unlike the other questions, my file list is rather small (about 200 lines), so using loops and RAM capacity
pose no problems.







shell-script text-processing






share|improve this question









New contributor




Enno is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|improve this question









New contributor




Enno is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|improve this question




share|improve this question








edited 2 days ago









Jeff Schaller

43.5k1161140




43.5k1161140






New contributor




Enno is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked 2 days ago









EnnoEnno

1483




1483




New contributor




Enno is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





Enno is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






Enno is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.












  • Does it have to be in bash proper, or just something available via the bash commandline? Other utilities are better positioned to process text.

    – Jeff Schaller
    2 days ago











  • @JeffSchaller Something accessible via the bash commandline. I was a bit unclear, sorry

    – Enno
    2 days ago












  • This is almost becoming a Code Golf :P

    – Richard de Wit
    yesterday






  • 1





    As a general rule, as long as you need to do something non-trivial, use your favourite scripting language over BASH. It will be less fragile (for example, against special characters or spaces), and much easier to expand whenever you need it (if you need three, or filter some of them away). Python or Perl should be installed in almost any Linux box, so they are good choices (unless you are working on embedded systems, like Busybox).

    – Davidmh
    yesterday

















  • Does it have to be in bash proper, or just something available via the bash commandline? Other utilities are better positioned to process text.

    – Jeff Schaller
    2 days ago











  • @JeffSchaller Something accessible via the bash commandline. I was a bit unclear, sorry

    – Enno
    2 days ago












  • This is almost becoming a Code Golf :P

    – Richard de Wit
    yesterday






  • 1





    As a general rule, as long as you need to do something non-trivial, use your favourite scripting language over BASH. It will be less fragile (for example, against special characters or spaces), and much easier to expand whenever you need it (if you need three, or filter some of them away). Python or Perl should be installed in almost any Linux box, so they are good choices (unless you are working on embedded systems, like Busybox).

    – Davidmh
    yesterday
















Does it have to be in bash proper, or just something available via the bash commandline? Other utilities are better positioned to process text.

– Jeff Schaller
2 days ago





Does it have to be in bash proper, or just something available via the bash commandline? Other utilities are better positioned to process text.

– Jeff Schaller
2 days ago













@JeffSchaller Something accessible via the bash commandline. I was a bit unclear, sorry

– Enno
2 days ago






@JeffSchaller Something accessible via the bash commandline. I was a bit unclear, sorry

– Enno
2 days ago














This is almost becoming a Code Golf :P

– Richard de Wit
yesterday





This is almost becoming a Code Golf :P

– Richard de Wit
yesterday




1




1





As a general rule, as long as you need to do something non-trivial, use your favourite scripting language over BASH. It will be less fragile (for example, against special characters or spaces), and much easier to expand whenever you need it (if you need three, or filter some of them away). Python or Perl should be installed in almost any Linux box, so they are good choices (unless you are working on embedded systems, like Busybox).

– Davidmh
yesterday





As a general rule, as long as you need to do something non-trivial, use your favourite scripting language over BASH. It will be less fragile (for example, against special characters or spaces), and much easier to expand whenever you need it (if you need three, or filter some of them away). Python or Perl should be installed in almost any Linux box, so they are good choices (unless you are working on embedded systems, like Busybox).

– Davidmh
yesterday










6 Answers
6






active

oldest

votes


















6














Use this command:



awk ' name[$1]++ 
END PROCINFO["sorted_in"] = "@ind_str_asc"
for (v1 in name) for (v2 in name) if (v1 < v2) print v1, v2
' files.dat


PROCINFO may be a gawk extension. 
If your awk doesn’t support it,
just leave out the PROCINFO["sorted_in"] = "@ind_str_asc" line
and pipe the output into sort (if you want the output sorted).



(This does not require the input to be sorted.)






share|improve this answer






























    7














    $ join -j 2 -o 1.1,2.1 file file | awk '!seen[$1,$2]++ && !seen[$2,$1]++'
    a b
    a c
    a d
    a e
    b c
    b d
    b e
    c d
    c e
    d e


    This assumes that no line in the input file contains any whitespace. It also assumes that the file is sorted.



    The join command creates the full cross product of the lines in the file. It does this by joining the file with itself on a non-existing field. The non-standard -j 2 may be replaced by -1 2 -2 2 (but not by -j2 unless you use GNU join).



    The awk command reads the result of this and only outputs results that are pairs that has not yet been seen.






    share|improve this answer

























    • What do you mean by "the file is sorted"? Sorted by which criteria?

      – Enno
      2 days ago











    • @Enno Sorted the way sort -b would sort it. join require sorted input files.

      – Kusalananda
      2 days ago



















    6














    If you have ruby installed:



    $ ruby -0777 -F'n' -lane '$F.combination(2) puts c.join(" ")' ip.txt
    a b
    a c
    a d
    a e
    b c
    b d
    b e
    c d
    c e
    d e



    • -0777 slurp entire file (should be okay as it is mentioned in OP that file size is small)


    • -F'n' split based on newline, so each line will be an element in $F array


    • $F.combination(2) generate combinations 2 elements at a time


    • puts c.join(" ") print as required

    • if input file can contain duplicates, use $F.uniq.combination(2)



    for 3 elements at a time:



    $ ruby -0777 -F'n' -lane '$F.combination(3) puts c.join(" ")' ip.txt
    a b c
    a b d
    a b e
    a c d
    a c e
    a d e
    b c d
    b c e
    b d e
    c d e





    With perl (not generic)



    $ perl -0777 -F'n' -lane 'for $i (0..$#F) 
    for $j ($i+1..$#F)
    print "$F[$i] $F[$j]n" ' ip.txt
    a b
    a c
    a d
    a e
    b c
    b d
    b e
    c d
    c e
    d e




    With awk



    $ awk ' a[NR]=$0 
    END for(i=1;i<=NR;i++)
    for(j=i+1;j<=NR;j++)
    print a[i], a[j] ' ip.txt
    a b
    a c
    a d
    a e
    b c
    b d
    b e
    c d
    c e
    d e





    share|improve this answer
































      6














      A python solution.
      The input file is fed to itertools.combinations from the standard library, which generates 2-length tuples that are formatted and printed to standard output.



      python3 -c 'from itertools import combinations
      with open("file") as f:
      lines = (line.rstrip() for line in f)
      lines = (" ".format(x, y) for x, y in combinations(lines, 2))
      print(*lines, sep="n")
      '





      share|improve this answer






























        3














        Here's one in pure shell.



        test $# -gt 1 || exit
        a=$1
        shift
        for f in "$@"
        do
        echo $a $f
        done
        exec /bin/sh $0 "$@"


        Example:



        ~ (137) $ sh test.sh $(cat file.dat)
        a b
        a c
        a d
        a e
        b c
        b d
        b e
        c d
        c e
        d e
        ~ (138) $





        share|improve this answer










        New contributor




        EdC is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.















        • 1





          Command substitution strips trailing newlines, so you're better off with something like <file.dat xargs test.sh than test.sh $(cat file.dat)

          – iruvar
          2 days ago


















        0














        Using Perl we can do it as shown:



        $ perl -lne '
        push @A, $_} exit
        a=$1
        shift
        for f in "$@"
        do
        echo $a $f
        done
        exec /bin/sh $0 "$@"


        Example:



        ~ (137) $ sh test.sh $(cat file.dat)
        a b
        a c
        a d
        a e
        b c
        b d
        b e
        c d
        c e
        d e
        ~ (138) $





        shareimprove this answer





















          share
          while ( @A )
          my $e = shift @A;
          print "$e $_" for @A;

          ' input.txt





          share{
          while ( @A )
          my $e = shift @A;
          print "$e $_" for @A;

          ' input.txt






          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered 2 days ago









          Rakesh SharmaRakesh Sharma

          342115




          342115




















              Enno is a new contributor. Be nice, and check out our Code of Conduct.









              draft saved

              draft discarded


















              Enno is a new contributor. Be nice, and check out our Code of Conduct.












              Enno is a new contributor. Be nice, and check out our Code of Conduct.











              Enno is a new contributor. Be nice, and check out our Code of Conduct.














              Thanks for contributing an answer to Unix & Linux Stack Exchange!


              • Please be sure to answer the question. Provide details and share your research!

              But avoid


              • Asking for help, clarification, or responding to other answers.

              • Making statements based on opinion; back them up with references or personal experience.

              To learn more, see our tips on writing great answers.




              draft saved


              draft discarded














              StackExchange.ready(
              function ()
              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f506815%2fbash-pair-each-line-of-file%23new-answer', 'question_page');

              );

              Post as a guest















              Required, but never shown





















































              Required, but never shown














              Required, but never shown












              Required, but never shown







              Required, but never shown

































              Required, but never shown














              Required, but never shown












              Required, but never shown







              Required, but never shown







              Popular posts from this blog

              Færeyskur hestur Heimild | Tengill | Tilvísanir | LeiðsagnarvalRossið - síða um færeyska hrossið á færeyskuGott ár hjá færeyska hestinum

              He _____ here since 1970 . Answer needed [closed]What does “since he was so high” mean?Meaning of “catch birds for”?How do I ensure “since” takes the meaning I want?“Who cares here” meaningWhat does “right round toward” mean?the time tense (had now been detected)What does the phrase “ring around the roses” mean here?Correct usage of “visited upon”Meaning of “foiled rail sabotage bid”It was the third time I had gone to Rome or It is the third time I had been to Rome

              Slayer Innehåll Historia | Stil, komposition och lyrik | Bandets betydelse och framgångar | Sidoprojekt och samarbeten | Kontroverser | Medlemmar | Utmärkelser och nomineringar | Turnéer och festivaler | Diskografi | Referenser | Externa länkar | Navigeringsmenywww.slayer.net”Metal Massacre vol. 1””Metal Massacre vol. 3””Metal Massacre Volume III””Show No Mercy””Haunting the Chapel””Live Undead””Hell Awaits””Reign in Blood””Reign in Blood””Gold & Platinum – Reign in Blood””Golden Gods Awards Winners”originalet”Kerrang! Hall Of Fame””Slayer Looks Back On 37-Year Career In New Video Series: Part Two””South of Heaven””Gold & Platinum – South of Heaven””Seasons in the Abyss””Gold & Platinum - Seasons in the Abyss””Divine Intervention””Divine Intervention - Release group by Slayer””Gold & Platinum - Divine Intervention””Live Intrusion””Undisputed Attitude””Abolish Government/Superficial Love””Release “Slatanic Slaughter: A Tribute to Slayer” by Various Artists””Diabolus in Musica””Soundtrack to the Apocalypse””God Hates Us All””Systematic - Relationships””War at the Warfield””Gold & Platinum - War at the Warfield””Soundtrack to the Apocalypse””Gold & Platinum - Still Reigning””Metallica, Slayer, Iron Mauden Among Winners At Metal Hammer Awards””Eternal Pyre””Eternal Pyre - Slayer release group””Eternal Pyre””Metal Storm Awards 2006””Kerrang! Hall Of Fame””Slayer Wins 'Best Metal' Grammy Award””Slayer Guitarist Jeff Hanneman Dies””Bullet-For My Valentine booed at Metal Hammer Golden Gods Awards””Unholy Aliance””The End Of Slayer?””Slayer: We Could Thrash Out Two More Albums If We're Fast Enough...””'The Unholy Alliance: Chapter III' UK Dates Added”originalet”Megadeth And Slayer To Co-Headline 'Canadian Carnage' Trek”originalet”World Painted Blood””Release “World Painted Blood” by Slayer””Metallica Heading To Cinemas””Slayer, Megadeth To Join Forces For 'European Carnage' Tour - Dec. 18, 2010”originalet”Slayer's Hanneman Contracts Acute Infection; Band To Bring In Guest Guitarist””Cannibal Corpse's Pat O'Brien Will Step In As Slayer's Guest Guitarist”originalet”Slayer’s Jeff Hanneman Dead at 49””Dave Lombardo Says He Made Only $67,000 In 2011 While Touring With Slayer””Slayer: We Do Not Agree With Dave Lombardo's Substance Or Timeline Of Events””Slayer Welcomes Drummer Paul Bostaph Back To The Fold””Slayer Hope to Unveil Never-Before-Heard Jeff Hanneman Material on Next Album””Slayer Debut New Song 'Implode' During Surprise Golden Gods Appearance””Release group Repentless by Slayer””Repentless - Slayer - Credits””Slayer””Metal Storm Awards 2015””Slayer - to release comic book "Repentless #1"””Slayer To Release 'Repentless' 6.66" Vinyl Box Set””BREAKING NEWS: Slayer Announce Farewell Tour””Slayer Recruit Lamb of God, Anthrax, Behemoth + Testament for Final Tour””Slayer lägger ner efter 37 år””Slayer Announces Second North American Leg Of 'Final' Tour””Final World Tour””Slayer Announces Final European Tour With Lamb of God, Anthrax And Obituary””Slayer To Tour Europe With Lamb of God, Anthrax And Obituary””Slayer To Play 'Last French Show Ever' At Next Year's Hellfst””Slayer's Final World Tour Will Extend Into 2019””Death Angel's Rob Cavestany On Slayer's 'Farewell' Tour: 'Some Of Us Could See This Coming'””Testament Has No Plans To Retire Anytime Soon, Says Chuck Billy””Anthrax's Scott Ian On Slayer's 'Farewell' Tour Plans: 'I Was Surprised And I Wasn't Surprised'””Slayer””Slayer's Morbid Schlock””Review/Rock; For Slayer, the Mania Is the Message””Slayer - Biography””Slayer - Reign In Blood”originalet”Dave Lombardo””An exclusive oral history of Slayer”originalet”Exclusive! Interview With Slayer Guitarist Jeff Hanneman”originalet”Thinking Out Loud: Slayer's Kerry King on hair metal, Satan and being polite””Slayer Lyrics””Slayer - Biography””Most influential artists for extreme metal music””Slayer - Reign in Blood””Slayer guitarist Jeff Hanneman dies aged 49””Slatanic Slaughter: A Tribute to Slayer””Gateway to Hell: A Tribute to Slayer””Covered In Blood””Slayer: The Origins of Thrash in San Francisco, CA.””Why They Rule - #6 Slayer”originalet”Guitar World's 100 Greatest Heavy Metal Guitarists Of All Time”originalet”The fans have spoken: Slayer comes out on top in readers' polls”originalet”Tribute to Jeff Hanneman (1964-2013)””Lamb Of God Frontman: We Sound Like A Slayer Rip-Off””BEHEMOTH Frontman Pays Tribute To SLAYER's JEFF HANNEMAN””Slayer, Hatebreed Doing Double Duty On This Year's Ozzfest””System of a Down””Lacuna Coil’s Andrea Ferro Talks Influences, Skateboarding, Band Origins + More””Slayer - Reign in Blood””Into The Lungs of Hell””Slayer rules - en utställning om fans””Slayer and Their Fans Slashed Through a No-Holds-Barred Night at Gas Monkey””Home””Slayer””Gold & Platinum - The Big 4 Live from Sofia, Bulgaria””Exclusive! Interview With Slayer Guitarist Kerry King””2008-02-23: Wiltern, Los Angeles, CA, USA””Slayer's Kerry King To Perform With Megadeth Tonight! - Oct. 21, 2010”originalet”Dave Lombardo - Biography”Slayer Case DismissedArkiveradUltimate Classic Rock: Slayer guitarist Jeff Hanneman dead at 49.”Slayer: "We could never do any thing like Some Kind Of Monster..."””Cannibal Corpse'S Pat O'Brien Will Step In As Slayer'S Guest Guitarist | The Official Slayer Site”originalet”Slayer Wins 'Best Metal' Grammy Award””Slayer Guitarist Jeff Hanneman Dies””Kerrang! Awards 2006 Blog: Kerrang! Hall Of Fame””Kerrang! Awards 2013: Kerrang! Legend”originalet”Metallica, Slayer, Iron Maien Among Winners At Metal Hammer Awards””Metal Hammer Golden Gods Awards””Bullet For My Valentine Booed At Metal Hammer Golden Gods Awards””Metal Storm Awards 2006””Metal Storm Awards 2015””Slayer's Concert History””Slayer - Relationships””Slayer - Releases”Slayers officiella webbplatsSlayer på MusicBrainzOfficiell webbplatsSlayerSlayerr1373445760000 0001 1540 47353068615-5086262726cb13906545x(data)6033143kn20030215029