Softpedia >Mac >Utilities >GNU Parallel >  Changelog

GNU Parallel Changelog

What's new in GNU Parallel 20150822

Aug 23, 2015
  • If $job->skip() is called in {= =} the job will not be run.
  • @arg can now be used in {= =}: parallel echo '{= $arg[2] < 5 and $_="j" =}' ::: 1 2 3 ::: 4 5 6
  • Version of GNU Parallel that uses PostgreSQL as backend https://github.com/stephen-fralich/parallel-sql/
  • A composite genome approach to identify phylogenetically informative data from next-generation sequencing http://www.biomedcentral.com/content/pdf/s12859-015-0632-y.pdf
  • SpeedSeq: ultra-fast personal genome analysis and interpretation http://www.nature.com/nmeth/journal/vaop/ncurrent/full/nmeth.3505.html
  • GNU Parallel is used in elPrep: https://github.com/ExaScience/elprep
  • Who actually reads the code? http://www.fsf.org/blogs/community/who-actually-reads-the-code
  • Introduction to GNU Parallel http://erictleung.com/2015/08/06/intro-to-gnu-parallel/
  • ¿Alguien se lee el código? http://www.taringa.net/post/linux/18854759/Alguien-se-lee-el-codigo.html
  • ¿Quién se lee el código fuente? https://victorhckinthefreeworld.wordpress.com/2015/08/12/quien-se-lee-el-codigo-fuente/
  • Distribution des traitements avec GNU Parallel http://blog.inovia-conseil.fr/?p=226#4
  • Эксперимент по проверке, читают ли код открытых проектов http://www.opennet.ru/opennews/art.shtml?num=42718
  • 多核心主機搭配 GNU parallel http://blog.zeroplex.tw/2015/08/gnu-parallel.html
  • Bug fixes and man page updates.

New in GNU Parallel 20150722 (Aug 23, 2015)

  • If a record fits in a block, --block only reads upto the size of --block. If not: --block reads a full --block more.
  • Due to widely spread distributions still using --tollef even after a year of being retired following a year of being obsolete, --tollef now gives an error unless used with --gnu.
  • --nice now uses a perl wrapper instead of the nice command.
  • Cookie 2013-08-18 was won by Mark Maimone, 1 year and 11 months after release.
  • A parser for GNU Parallel --joblog files: https://github.com/harryjubb/joblog
  • GNU Parallel was cited in: Dynamical formation & scattering of hierarchical triples: Cross sections, Kozai-Lidov oscillations, and collisions http://arxiv.org/pdf/1507.03593.pdf
  • GNU Parallel was cited in: A Fast Compiler for NetKAT http://arxiv.org/pdf/1506.06378.pdf
  • GNU Parallel was cited in: Flexible Modeling of Epidemics with an Empirical Bayes Framework http://www.stat.cmu.edu/~ryantibs/papers/empbayes.pdf
  • GNU Parallel was cited in: Decomposing Time-Lapse Paintings into Layers http://cs.gmu.edu/~ygingold/timemap/Decomposing Time-Lapse Paintings into Layers (Tan et al 2015 SIGGRAPH) 600dpi.pdf
  • GNU Parallel was used in: Somatic mutation calling from the matched tumor-normal exome-sequencing http://fafner.meb.ki.se/biostatwiki/somac/
  • Parallel Processing from the Command Line https://www.perpetual-beta.org/weblog/parallel-processing-from-the-command-line.html
  • Using GNU Parallel to speed up network operations http://vielmetti.typepad.com/vacuum/2015/06/using-gnu-parallel-to-speed-up-network-operations.html
  • Crawl faster with "parallel" - but how fast? http://andreas-wpv.blogspot.com/2015/07/crawl-faster-with-parallel-but-how-fast.html
  • Simultaneously benchmark many URLs with ApacheBench and GNU parallel https://www.simonholywell.com/post/2015/06/parallel-benchmark-many-urls-with-apachebench/
  • Bug fixes and man page updates.

New in GNU Parallel 20150622 (Jun 24, 2015)

  • --halt has been rewritten completely. You can now combine percentages with success or fail. See the man page.
  • Exit values 102..254 have been removed. 101 means more than 100 jobs failed.
  • Killing through --timeout, --memfree, or --halt is now done as a process group.
  • --termseq determines which signals are sent when a job is killed.
  • An empty argument would previously cause no string to be inserted. This is now changed to '' being inserted, thus prepending a space to the output of: parallel echo {} b ::: ''
  • $PARALLEL_ENV can now be set to an environment prepending the command. Used in env_parallel as mentioned in the manpage.
  • --retry-failed will retry all failed jobs in a joblog. It will ignore any command given.
  • --ssh and $PARALLEL_SSH can be used to set the command used for ssh. The command is assumed to behave the same way as ssh.
  • --fifo now works in csh, too.
  • Q(...) can be used in {= =} to shell quote a string.
  • Bug fixes and man page updates.

New in GNU Parallel 20150522 (May 22, 2015)

  • Security: The security issue for --sshlogin + --fifo/--cat has been fixed. Thereby all issues with http://lists.gnu.org/archive/html/parallel/2015-04/msg00045.html have been fixed.
  • Security: After further security analysis the issue fixed in 20150422 also fixed the problem for --tmux.
  • GNU Parallel was cited in: CIDER: a pipeline for detecting waves of coordinated transcriptional regulation in gene expression time-course data http://biorxiv.org/content/biorxiv/early/2015/03/17/012518.full.pdf
  • GNU Parallel was cited in: Building Genomic Analysis Pipelines in a Hackathon Setting with Bioinformatician Teams: DNA-seq, Epigenomics, Metagenomics and RNA-seq http://biorxiv.org/content/biorxiv/early/2015/05/05/018085.full.pdf
  • GNU Parallel was cited in: Toward Enhanced Metadata Quality of Large-Scale Digital Libraries: Estimating Volume Time Range https://www.ideals.illinois.edu/bitstream/handle/2142/73656/186_ready.pdf
  • GNU Parallel was cited in: Sequencing the cap-snatching repertoire of H1N1 influenza provides insight into the mechanism of viral transcription initiation http://nar.oxfordjournals.org/content/early/2015/04/20/nar.gkv333.full.pdf
  • GNU Parallel was cited in: Genome assembly using Nanopore-guided long and error-free DNA reads http://www.biomedcentral.com/content/pdf/s12864-015-1519-z.pdf
  • GNU Parallel was cited in: Contrasting regional architectures of schizophrenia and other complex diseases using fast variance components analysis http://biorxiv.org/content/biorxiv/early/2015/03/13/016527.full.pdf
  • GNU Parallel was used (unfortunately with wrong citation) in: Comparing the CarbonTracker and TM5-4DVar data assimilation systems for CO2 surface flux inversions http://www.atmos-chem-phys-discuss.net/15/8883/2015/acpd-15-8883-2015-discussion.html
  • GNU Parallel was used in: Gene Set Omic Analysis (GSOA) method https://bitbucket.org/srp33/gsoa
  • A Quick and Neat :) Orchestrator using GNU Parallel http://www.elsotanillo.net/2015/05/a-quick-and-neat-orchestrator-using-gnu-parallel/
  • Execute commands on multiple computers using GNU Parallel (setting up a cluster on the cheap) https://spectraldifferences.wordpress.com/2015/04/26/execute-commands-on-multiple-computers-using-gnu-parallel-setting-up-a-cluster-on-the-cheap/
  • Functions and GNU parallel for effective cluster load management http://genomespot.blogspot.dk/2015/04/functions-and-gnu-parallel-for.html
  • Use parallel processing to save time importing databases http://drupalsun.com/node/41854
  • Run multiple ssh commands in parallel with GNU Parallel http://www.ameir.net/blog/archives/380-run-multiple-ssh-commands-in-parallel-with-gnu-parallel.html
  • Importing huge databases faster https://www.lullabot.com/blog/article/importing-huge-databases-faster
  • Run multiple ssh commands in parallel with GNU Parallel https://www.ameir.net/blog/archives/380-run-multiple-ssh-commands-in-parallel-with-gnu-parallel.html/comment-page-1
  • Parallel? Gnu parallel! https://debian.pro/1834
  • Bug fixes and man page updates.

New in GNU Parallel 20150422 (Apr 25, 2015)

  • Security fix. An attacker on the local system could make you overwrite one of your own files with a single byte. The problem exist when you use --compress or --tmux or --pipe or --cat or --fifo. The attacker must figure out the randomly chosen file name and create a symlink within a time window of 15 ms.
  • --shuf will generate all jobs, and shuffle them before running them. This is useful to get a quick preview of the results before running the full batch.
  • GNU Parallel now has a DOI: https://dx.doi.org/10.5281/zenodo.16303
  • GNU Parallel was cited in: Scaling Machine Learning for Target Prediction in Drug Discovery using Apache Spark https://cris.cumulus.vub.ac.be/portal/files/5147244/spark.pdf
  • GNU Parallel was cited in: Bayesian inference of protein structure from chemical shift data https://peerj.com/articles/861/
  • GNU Parallel was cited in: Toward Enhanced Metadata Quality of Large-Scale Digital Libraries: Estimating Volume Time Range https://www.ideals.illinois.edu/bitstream/handle/2142/73656/186_ready.pdf
  • GNU Parallel was cited in: Contrasting regional architectures of schizophrenia and other complex diseases using fast variance components analysis http://biorxiv.org/content/biorxiv/early/2015/03/13/016527.full.pdf
  • GNU Parallel was cited in: Experimental study on the Wind Farm Substation Cable Installation Problem http://i11www.iti.uni-karlsruhe.de/_media/teaching/theses/ma-schmitz-14.pdf
  • GNU Parallel was used in: landsat-gifworks https://github.com/KAPPS-/landsat-gifworks
  • GNU Parallel was used in: https://github.com/exascience/elprep
  • GNU Parallel was used in: http://search.cpan.org/~jillrowe/HPC-Runner-GnuParallel-0.01/
  • GNU Parallel was used in: http://search.cpan.org/~ajpage/Bio-Roary-2.0.7/lib/Bio/Roary/JobRunner/Parallel.pm
  • GNU Parallel was used in: Rapid haploid SNP calling https://github.com/tseemann/snippy
  • Pictures and Metadata http://www.ozzy.no/2015/02/05/pictures-and-metadata/
  • Task automation with bash and parallel https://biowize.wordpress.com/2015/03/23/task-automation-with-bash-and-parallel/
  • How To: Speed Up File Transfers in Linux using RSync with GNU Parallel http://www.yourownlinux.com/2015/04/speed-up-file-transfers-using-rsync-with-gnu-parallel.html
  • Mirroring a Git Repository https://avacariu.me/articles/mirroring-a-git-repository.html
  • Add an ENVI header to JAXA Global Mangrove Watch PALSAR tiles https://spectraldifferences.wordpress.com/2015/03/24/add-an-envi-header-to-jaxa-global-mangrove-watch-palsar-tiles/
  • GNU Parallel (Sebuah Uji Coba) http://kaka.prakasa.my.id/2014/09/04/gnu-parallel-sebuah-uji-coba/
  • Bug fixes and man page updates.
  • GNU Parallel - For people who live life in the parallel lane.

New in GNU Parallel 20150322 (Mar 23, 2015)

  • New in this release:
  • --number-of-cores respects 'taskset' on GNU/Linux.
  • --joblog --pipe gives the data send and received in the log.
  • GNU Parallel was tested to support 100 GB sized records in --pipe. A few bugs was fixed to support >2 GB records. It works, but is rather slow.
  • GNU Parallel was cited in: RIG: Recalibration and Interrelation of genomic sequence data with the GATK http://www.g3journal.org/content/early/2015/02/13/g3.115.017012.full.pdf+html
  • GNU Parallel was cited in: MPI-blastn and NCBI-TaxCollector: Improving metagenomic analysis with high performance classification and wide taxonomic attachment http://www.worldscientific.com/doi/abs/10.1142/S0219720014500139?af=R&
  • GNU Parallel was cited in: Recent evolution in Rattus norvegicus is shaped by declining effective population size http://biorxiv.org/content/biorxiv/early/2015/03/01/015818.full.pdf
  • GNU Parallel was cited in: Evidence for DCO+ as a probe of ionization in the warm disk surface http://arxiv.org/pdf/1503.02659.pdf
  • GNU Parallel was cited in: De novo assembly and annotation of the Asian tiger mosquito (Aedes albopictus) repeatome with dnaPipeTE from raw genomic reads and comparative analysis with the yellow fever mosquito (Aedes aegypti) http://gbe.oxfordjournals.org/content/early/2015/03/11/gbe.evv050.full.pdf
  • GNU Parallel was cited in: A General Approach to Network Configuration Analysis http://research.microsoft.com/en-us/um/people/ratul/papers/nsdi2015-batfish.pdf
  • GNU Parallel was cited in: Scrimer: designing primers from transcriptome data http://onlinelibrary.wiley.com/doi/10.1111/1755-0998.12403/pdf
  • GNU Parallel was cited in: Efficient Retrieval of Key Material for Inspecting Potentially Malicious Traffic in the Cloud http://sacko.uk/pdf/2015.1.pdf
  • GNU Parallel will be presented at Strataconf: Poor Man's Parallel Pipelines http://strataconf.com/big-data-conference-uk-2015/public/schedule/detail/40031
  • GNU Parallel was used in: https://github.com/alexbyrnes/FCC-Political-Ads_The-Code
  • GNU Parallel was used in: https://github.com/martymac/fpart
  • GNU Parallel was used in: https://github.com/hoytak/diabetic-retinopathy-code
  • GNU Parallel was used in: https://github.com/mehmattski/HybSeqPipeline
  • GNU Parallel was used in: http://search.cpan.org/~ajpage/Bio-Roary-2.0.0/lib/Bio/Roary/JobRunner/Parallel.pm
  • Using GNU Parallel on a Raspberry Pi cluster: http://www.dcglug.org.uk/cluster-progress/
  • An introduction to vector tiling and map reduce in postgis: http://dimensionaledge.com/intro-vector-tiling-map-reduce-postgis/
  • Running scripts in parallel with GNU Parallel: http://code.jasonbhill.com/2015/03/
  • Mahout: Parallelising the creation of DecisionTrees: http://www.markhneedham.com/blog/2012/12/27/mahout-parallelising-the-creation-of-decisiontrees/
  • High-Performance Scientific Computing: Running serial jobs in parallel https://support.scinet.utoronto.ca/education/staticpublic/course178/serial.pdf
  • Parallel cardinality on your laptop https://highonscience.wordpress.com/2015/03/15/parallel-cardinality-on-your-laptop/
  • Experiments about a better locate using grep http://a3nm.net/blog/better_locate.html
  • Homework with GNU Parallel: https://support.scinet.utoronto.ca/education/staticpublic/course178content349.html
  • Parallel - Jobs in Skripten parallelisieren: https://slzm.de/blog/linux-tool-des-tages-parallel-jobs-in-skripten-parallelisieren/#more-612
  • GNU Parallel, czyli 100% użycia procesora http://matmatyk.blogspot.dk/2015/03/gnu-parallel-czyli-100-uzycia-procesora.html
  • Bug fixes and man page updates.

New in GNU Parallel 20150222 (Feb 23, 2015)

  • --tmux has gotten a major overhaul.
  • GNU Parallel was cited in: RaftLib: A C++ Template Library for High
  • Performance Stream Parallel Processing
  • http://www.cs.wustl.edu/~lip/pubs/pmam15_jbeard.pdf
  • GNU Parallel was cited in: Towards Collaborative Exploration and
  • Analysis of Big Data from Mars: A Noachis Terra Case Study
  • http://link.springer.com/chapter/10.1007/978-3-319-13865-7_25
  • GNU Parallel was cited in: Quantifying properties of hot and dense
  • QCD matter through systematic model-to-data comparison
  • http://arxiv.org/pdf/1502.00339.pdf
  • GNU Parallel was cited in: Towards Collaborative Exploration and
  • Analysis of Big Data from Mars: A Noachis Terra Case Study
  • http://link.springer.com/chapter/10.1007/978-3-319-13865-7_25
  • GNU Parallel was cited in: Towards Recommender Engineering Tools and
  • Experiments for Identifying Recommender Differences
  • http://elehack.net/research/thesis/mde-thesis.pdf
  • GNU Parallel was using (unfortunately with wrong citation) in:
  • Performance and Scaling Comparison Study of RDBMS and NoSQL
  • (MongoDB)
  • http://ijact.in/wp-content/uploads/2014/11/COMPUSOFT-311-1270-1275.pdf
  • GNU Parallel was used (unfortunately without citation) in: Parallel
  • Implementation of Big Data Pre-Processing Algorithms for Sentiment
  • Analysis of Social Networking Data
  • http://www.researchmathsci.org/IJFMAart/ijfma-v6n2-7.pdf
  • GNU Parallel was used (unfortunately without citation) in: SpeedSeq:
  • Ultra-fast personal genome analysis and interpretation
  • http://biorxiv.org/content/biorxiv/early/2014/12/05/012179.full.pdf
  • Zip Folders with GNU Parallel
  • http://fazky.github.io/Linux/2015-01-07-GNU-Parallel.html
  • Using GNU Parallel with Freesurfer
  • http://programminginadarkroom.blogspot.dk/2015/02/using-gnu-parallel-with-freesurfer.html
  • GNU Parallel is used in Velociraptor: https://github.com/ericwhyne/Velociraptor
  • Marcus Beach GNU Parallel http://marcusbeach.co/gnu-parallel/
  • Bug fixes and man page updates.

New in GNU Parallel 20150122 (Jan 22, 2015)

  • Remote jobs now send stderr (standard error) to stderr (standard error) instead of stdout (standard output).
  • Remote execution command is now packed using base64 encoded bzip2. This means that big environments (app. 100 KB) can be
  • transferred. This change makes remote execution alpha quality.
  • --semaphoretimeout now takes a negative value. This means give up if the semaphore is not gotten within the timeout.
  • --halt -1 and -2 now means halt if a job succeeds (so the opposite of 1 and 2).
  • Bash's second fix of shellshock caused --env to break again when exporting functions. This has been fixed again.
  • A semibig refactoring of big functions. All non-trivial functions are now less than 100 lines. The refactoring makes this release beta quality.
  • A description of the design decisions for GNU Parallel can be found in 'man parallel_design'.
  • A bug fix in replacement strings cause rewrite of the replacement function. This makes use of replacement strings alpha quality.
  • GNU Parallel was cited in: SlideToolkit: An Assistive Toolset for the Histological Quantification of Whole Slide Images http://www.plosone.org/article/info:doi/10.1371/journal.pone.0110289
  • GNU Parallel was cited in: Improving Energy and Performance with Spintronics Caches in Multicore Systems http://link.springer.com/chapter/10.1007/978-3-319-14313-2_24
  • GNU Parallel was cited in: Parallel post-processing with MPI-Bash http://dl.acm.org/citation.cfm?id=2691137
  • GNU Parallel was cited in: Distinguishing cause from effect using observational data: methods and benchmarks http://arxiv-web3.library.cornell.edu/pdf/1412.3773.pdf
  • GNU Parallel was cited in: Data Science at the Command Line http://cdn.oreillystatic.com/oreilly/booksamplers/9781491947852_sampler.pdf
  • GNU Parallel was cited in: Resolution of Genetic Map Expansion Caused by Excess Heterozygosity in Plant Recombinant Inbred Populations http://www.g3journal.org/content/4/10/1963.full.pdf
  • GNU Parallel was cited in: Bayesian Inference of Protein Structure from Chemical Shift Data https://peerj.com/preprints/692v1.pdf
  • GNU Parallel was cited in: Clonal interference and Mullerʼs ratchet in spatial habitats http://iopscience.iop.org/1478-3975/11/5/056003
  • GNU Parallel was cited in: Web-scale content reuse detection ftp://info.isi.edu/isi-pubs/tr-692.pdf
  • GNU Parallel was cited in: Computação Verde na Camada de Aplicação https://linux.ime.usp.br/~tpaiva/mac0499/files/monografia.pdf
  • GNU Parallel was cited in: Frameshift alignment: statistics and post-genomic applications http://bioinformatics.oxfordjournals.org/content/30/24/3575.full.pdf+html
  • GNU Parallel was cited in: Towards Recommender Engineering Tools and Experiments for Identifying Recommender Differences http://elehack.net/research/thesis/mde-thesis.pdfNU Parallel was cited in: Speaker Environment Classification Using Rhythm Metrics in Levantine Arabic Dialect
  • http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6923918
  • GNU Parallel was cited in: A Formal Proof Of The Kepler Conjecture http://arxiv.org/pdf/1501.02155.pdf
  • GNU Parallel was cited in: Allocation Policy Analysis for Cache Coherence Protocols for STT-MRAM-based caches http://conservancy.umn.edu/bitstream/handle/11299/169362/Nandkar_umn_0130M_15551.pdf
  • GNU Parallel was cited in: Using Trusted Platform Module for securing virtual environment access in Cloud http://dar.aucegypt.edu:8080/bitstream/handle/10526/4237/Using Trusted Platform Module for Securing Virtual Environment Access in Cloud.pdf
  • GNU Parallel was cited in: Weakly Supervised Learning for Compositional Sentiment Recognition http://computerlinguist.org/static/downloads/ma-thesis/Master_Thesis-Michael_Haas-Weakly_Supervised_Learning_for_Compositional_Sentiment_Recognition.pdf
  • GNU Parallel was cited in: Agility and Performance in Elastic Distributed Storage http://dl.acm.org/citation.cfm?id=2668129
  • GNU Parallel was used (unfortunately without citation) in:
  • Large Scale Author Name Disambiguation in Digital Libraries
  • http://semanticommunity.info/@api/deki/files/31523/P227_3546.pdf
  • GNU Parallel is used in BEDOPS: https://media.readthedocs.org/pdf/bedops/v2p5p0/bedops.pdf
  • Job ad asking for GNU Parallel skills: http://www.aplitrak.com/?adid=dXMzMTYxMjQuMzg0MTkuMTU1MEBtYW5wb3dlcmZveG5hLmFwbGl0cmFrLmNvbQ#sthash.kYCIhhty.KcoT6HnJ.dpbs
  • GNU Parallel: Open Source For You (OSFY) magazine, October 2013 edition http://www.shakthimaan.com/posts/2014/11/27/gnu-parallel/news.html
  • Multithreaded Encryption and Compression http://www.krazyworks.com/multithreaded-encryption-and-compression/
  • GNU Parallel on OSX and Linux https://darknightelf.wordpress.com/2015/01/01/gnu-parallel-on-osx/
  • How to build gnu parallel from source on Ubuntu PC? http://blog.ovidiuparvu.com/build-gnu-parallel-source-ubuntu-pc/
  • GNU parallel for openaddr-process-one https://nelsonslog.wordpress.com/2015/01/09/gnu-parallel-for-openaddr-process-one/
  • From SRA Project to FastQ http://danielecook.com/sra-project-fastq/
  • Software Development Tooling http://www.computer.org/csdl/mags/so/2014/06/mso2014060021.pdf
  • コマンドを並列に実行するGNU parallelがとても便利 http://bicycle1885.hatenablog.com/entry/2014/08/10/143612
  • Bug fixes and man page updates.

New in GNU Parallel 20141122 (Nov 24, 2014)

  • CHANGES:
  • Remote systems can be divided into hostgroups (e.g. web and db) by prepending '@groupname/' to the sshlogin. Multiple groups can be given by separating groups with '+'. E.g. @web/www1 @web+db/www2 @db/mariadb
  • Remote execution can be restricted to servers that are part of one or more groups by '@groupname' as an sshlogin. Multiple groups can be given by separating groups with '+'. E.g. -S @web or -S @db+web
  • With --hostgroup you can restrict arguments to certain hostgroups by appending '@groupname' to the argument. Multiple groups can be given by separating groups with '+'. E.g. my_web_arg@web db-or-web-arg@db+web db-only-arg@db Thanks to Michel Courtine for developing a prototype for this.
  • GNU Parallel was cited in: SlideToolkit: An Assistive Toolset for the Histological uantification of Whole Slide Images http://www.plosone.org/article/info:doi/10.1371/journal.pone.0110289#close
  • GNU Parallel was cited in: Exploring a multiprocessor design space to analyze the impact of using STT-RAM in the memory hierarchy http://conservancy.umn.edubitstream/handle/11299/167286/Borse_umn_0130M_15431.pdf
  • Command-Line OCR with Tesseract on Mac OS X https://ryanfb.github.io/etc/2014/11/13/command_line_ocr_on_mac_os_x.html
  • Bug fixes and man page updates.

New in GNU Parallel 20141022 (Oct 23, 2014)

  • --env was changed to deal with Bash's new way of exporting a function.
  • GNU Parallel was cited in: Exclusion of cosmic rays in protoplanetary disks. II. Chemical gradients and observationalsignatures http://iopscience.iop.org/0004-637/794/2/123/article
  • GNU Parallel was cited in: Application of Machine Learning toAlgorithm Selection for TSPhttp://www.dbai.tuwien.ac.at/staff/musliu/art_ictai_cam.pdf
  • GNU Parallel was cited in: Performance Models for LTE-AdvancedRandom Accesshttp://repository.asu.edu/attachments/137242/conten/Tyagi_asu_0010E_14116.pdf
  • GNU Parallel was used (unfortunately without citation) in: De NovoInsertions and Deletions of Predominantly Paternal Origin AreAssociated with Autism Spectrum Disorderhttp://www.cell.com/cms/attachment/2019079667/2039310868/mmc1.pdf
  • GNU Parallel was determined 'popular' by Statistical Analysis ofPopular Open Source Software Projects and Their Communitieshttp://andiwre.itmaranatha.org/prosidin/ICITEE2014/PaperAndiWRE-ICITEE2014.pdf
  • GNU Parallel is demonstrated in: Data Science at the Command Line:Facing the Future with Time-Tested Tools
  • GNU Parallel was covered in a talk at Sydney Perl Mongers grouphttp://www.openfusion.net/talks/pwp/#24
  • GNU Parallel and how to get started with ithttp://www.jduck.net/blog/2014/09/3/gnu-paralell/
  • Behat/Selenium/Parallelhttp://jaffamonkey.com/behat-selenium-parallel/
  • Comparing Golang, Scala, Elixir and Ruby for ETLhttp://www.dimroc.com/2014/0/29/etl-language-showdown/
  • Parallelizing Batch Jobs for Fun and Profithttp://mikeseidle.com/tech/programmin/2013/03/parallelizing-batch-jobs
  • Assemble a list of which of these hosts are up or downhttp://opsfun.wordpress.co/2014/03/31/gnu-parallel/
  • Running commands on many nodes simultaneouslyhttp://opsfun.wordpress.co/2014/03/31/19/
  • Ubuntu 13.04でGNU ParallelをGNUモードで動かすhttp://qiita.co/YusukeSuzuki@github/items/7b96b5876bf980f21e85
  • Bug fixes and main page updates.

New in GNU Parallel 20140822 (Aug 23, 2014)

  • GNU Parallel now uses the same shell it was started from as the command shell for local jobs. So if GNU Parallel is started from tcsh it will use tcsh as its shell even if the login $SHELL is different. For remote jobs the login $SHELL will be used.
  • The whole current environment in bash can be copied by using a shell wrapper function (Search manual for env_parallel).
  • --plus adds the replacement strings {+/} {+.} {+..} {+...} {..} {...} {/..} {/...}. The idea being that '+foo' matches the opposite of 'foo' and {} = {+/}/{/} = {.}.{+.} = {+/}/{/.}.{+.} = {..}.{+..} = {+/}/{/..}.{+..} = {...}.{+...} = {+/}/{/...}.{+...}
  • GNU Parallel now deals correctly with the combination rsync-3.1.X-client and rsync-2.5.7-server
  • GNU Parallel was cited in: A Web Service for Scholarly Big Data Information Extraction http://patshih.ist.psu.edu/publications/Williams-CiteSeerExtractor-ICWS14.pdf
  • Comparison of the speed of different GNU Parallel versions: http://lists.gnu.org/archive/html/parallel/2014-08/msg00030.html
  • GNU Parallel was covered in the webcast 2014-08-20: Data Science at the Command Line http://www.oreilly.com/pub/e/3115
  • Distributed processing with GNU parallel http://kazjote.eu/2014/08/11/distributed-processing-with-gnu-parallel
  • A Peek into GNU Parallel http://blog.dataweave.in/post/94238943763/a-peek-into-gnu-parallel
  • Сборка GNU parallel для CentOS/RHEL http://www.stableit.ru/2014/07/gnu-parallel-centosrhel.html
  • Bug fixes and man page updates.

New in GNU Parallel 20140722 (Jul 22, 2014)

  • {= perl expression =} can be used as replacement string. The expression should modify $_. E.g. {= s/\.gz$// =} to remove .gz from the string. This makes replacement strings extremely flexible.
  • Positional perl expressions (similar to {2}) are given as {=2 perl expression=} where 2 is the position.
  • One small backwards incompatability: {1}_{2} will replace {2} with the empty string if there is only one argument. Previously {2} would have been left untouched.
  • Replacement strings can be defined using --rpl. E.g. parallel --rpl '{.gz} s/\.gz$//' echo {.gz} ::: *.gz
  • The parenthesis around {= perl expression =} can be changed with --parens.
  • --tmux will direct the output to a tmux session instead of files. Each running jobs will be in its own window.
  • --halt 10% will stop spawning new jobs if 10% failed so far.
  • Bug fixes and man page updates.

New in GNU Parallel 20140622 (Jun 23, 2014)

  • --shebang and --shebang-wrap now works on FreeBSD.
  • --pipepart works with --header, --regexp, --cat and --fifo.
  • ./configure --disable-documentation will not build documentation.
  • {%} works as job slot.
  • GNU Parallel was cited in: Energy Efficient, High-speed Communication in WSNs https://gupea.ub.gu.se/bitstream/2077/35801/1/gupea_2077_35801_1.pdf
  • GNU Parallel was cited in: Ferroelectric contributions to anomalous hysteresis in hybrid perovskite solar cells http://arxiv.org/pdf/1405.5810.pdf
  • Processes Paralleling to Speed up Computing and Tasks Execution in Linux http://kukuruku.co/hub/nix/processes-paralleling-to-speed-up-computing-and-tasks-execution-in-linux
  • Moving / Copying lots of s3 files quickly using gnu parallel http://blog.aclarke.eu/moving-copying-lots-of-s3-files-quickly-using-gnu-parallel/
  • Speeding up grep log queries with GNU Parallel http://www.tripwire.com/state-of-security/incident-detection/speeding-grep-queries-gnu-parallel/
  • Export quicklooks with vector overlay using TuiView http://spectraldifferences.wordpress.com/2014/03/08/export-quicklooks-with-vector-overlay-using-tuiview/
  • Bug fixes and man page updates.

New in GNU Parallel 20140522 (May 23, 2014)

  • Use --semaphoretimeout to ignore the semaphore lock after a while.
  • {%} introduced as job slot replacement string. It has known bugs.
  • --slotreplace changes slot replacement string.
  • Bug fixes and man page updates.

New in GNU Parallel 20140422 (Apr 24, 2014)

  • --pipepart is a highly efficient alternative to --pipe if the input is a real file and not a pipe.
  • If using --cat or --fifo with --pipe the {} in the command will be replaced with the name of a physical file and a fifo respectively containing the block from --pipe. Useful for commands that cannot read from standard input (stdin).
  • --controlmaster has gotten an overhaul and is no longer experimental.
  • --env is now copied when determining CPUs on remote system. Useful for copying $PATH if parallel is not in the normal path.
  • --results now chops the argument if the argument is longer than the allowed path length.
  • Build now survives if pod2are not installed.
  • The git repository now contains tags of releases.
  • Bug fixes and man page updates.

New in GNU Parallel 20140322 (Mar 27, 2014)

  • Bug fixes and man page updates.

New in GNU Parallel 20140222 (Feb 25, 2014)

  • --tollef has been retired.
  • --compress has be redesigned due to bugs.
  • Format of --eta and --joblog has changed slightly.
  • Bug fixes and man page updates.

New in GNU Parallel 20140122 (Jan 23, 2014)

  • Bug fixes and man page updates.

New in GNU Parallel 20131222 (Dec 27, 2013)

  • Detect if the buffer dir $TMPDIR runs out of space. This changes a central piece of code, making this release alpha quality.
  • --keep-order --(n)onall will sort according to the sshlogin.
  • Detection number of CPUs on DEC Tru64.
  • GNU sql: --list-tables for sqlite3.
  • Bug fixes and man page updates.

New in GNU Parallel 20131122 (Nov 25, 2013)

  • A citation notice is printed on stderr only if stderr is a terminal, the user has not specified --no-notice and the user has not run --bibtex once. This makes the release alpha quality.
  • --compress will compress temporary files. If the output is big and very compressible this will take up less disk space in $TMPDIR and possibly be faster due to less disk I/O.
  • --compress-program comtrols which program to use for compressing temporary files.
  • --bar show progress as a progress bar compatible with zenity.
  • --resume can now be used with --result: Jobs already run will be skipped.
  • --transfer and --basefile support paths relative to the --workdir by inserting /./ into the path.
  • Bug fixes and man page updates.

New in GNU Parallel 20131022 (Oct 22, 2013)

  • --transfer files with /./ in the path will copy the files relative to the --workdir.
  • The maximal command length is now cached in a file halfing the startup time.
  • Bug fixes and man page updates.

New in GNU Parallel 20130922 (Sep 23, 2013)

  • PDF-files of documentation is now included.
  • Bug fixes and man page updates.

New in GNU Parallel 20130822 (Aug 23, 2013)

  • A tutorial showing much of GNU Parallel's functionality. Spend an hour walking through the tutorial. Your commandline will love you for it. man parallel_tutorial
  • --line-buffer will buffer output on line basis. --group keeps the output together for a whole job. --ungroup allows output to mixup with half a line coming from one job and half a line coming from another job. --line-buffer fits between these two; it prints a full line, but will allow for mixing lines of different jobs.
  • --record-env records all variables and functions set. Useful to record the normal environment before using --env _.
  • --env _ exports all functions and variables to the remote server except for the ones recorded.
  • New signing key. Due to recommendations from NIST http://www.keylength.com/en/4/ the signing key was changed from 1024D/ID:FFFFFFF1 to 9888R/ID:88888888.
  • Agalma: an automated phylogenomics workflow http://arxiv.org/pdf/1307.6432
  • Job ad asking for GNU Parallel expertise http://seattle.craigslist.org/est/sof/4006079352.html
  • Transient Beowulf Clustering with GNU Parallel and SSHfs http://www.reddit.com/r/linux/comments/1ka8mn/transient_beowulf_clustering_with_gnu_parallel/
  • Aligning to unique regions http://davetang.org/muse/2013/07/22/aligning-to-unique-regions/
  • Top 10 scripting tricks for basic bioinformatics http://students.washington.edu/bowmanjs/wordpress/?p=873
  • Using GNU Parallel to speed up and simplify data analyzes http://www.sergeymarkov.com/blog/2013/07/using-gnu-parallel-to-speed-up-and-simplify-data-analyzes/
  • Compression of files in parallel using GNU parallel http://codextechnicanum.blogspot.dk/2013/07/compression-of-files-in-parallel-using.html
  • Using GNU Parallel to roll-your-own Map Reduce! http://www.rankfocus.com/?p=1
  • Using GNU Parallel with Amazon media.amazonwebservices.com/AWS_Amazon_EMR_Best_Practices.pdf
  • Some useful comments on GNU Parallel https://news.ycombinator.com/item?id=6209767
  • Using GNU Parallel to count reads mapping to genes for multiple BAMs http://drduanehassane.com/blog/sequencing-resources
  • TimeMachineっぽいバックアップスクリプト http://rio.tc/2013/07/timemachine-1.html
  • GNU ParallelでAndroid NDKの全バージョンを一括ダウンロード http://qiita.com/mazgi/items/b10bf0ff3da2045d19ab
  • Bug fixes and man page updates.

New in GNU Parallel 20130722 (Jul 23, 2013)

  • --round-robin with --pipe will write all blocks to already running jobs.
  • --env can now transfer Bash function for remote execution.
  • Quite a few bug fixes and man page updates

New in GNU Parallel 20130622 (Jun 24, 2013)

  • --xapply now recycles arguments if an input source has morearguments than others.
  • The sleep time between jobs is now both increased and decreasedexponentially.
  • 10 seconds installation check the signature using GnuPG if GnuPG isinstalled.
  • Developer job asking for GNU Parallel expertise.http://careers.stackoverflow.com/jobs/35562/developer-big-data-geo-and-web-climate-central
  • A small utility program to run youtube-dl in parallel.https://github.com/dlh/youtube-dl-parallel
  • Parallelizing Freesurfer:http://blog.cogneurostats.com/?p=148
  • Bug fixes and man page updates.

New in GNU Parallel 20130522 (May 22, 2013)

  • --ctrlc is now default if --pipe is not used; thus making it possible to kill remotely started jobs simply by pressing Ctrl-C.
  • --timeout 200% now means kill jobs that take more than twice the time of the median run time of a job.
  • Jobs are now distributed round robin when having mulitiple --sshlogin instead of filling up one --sshlogin at a time.
  • niceload: darwin version of --io --mem --swap. Thanks to Anders F Björklund.
  • Bug fixes and man page updates.

New in GNU Parallel 20130422 (Apr 29, 2013)

  • HPUX CPU/core detection. Thanks to Javier Tarradas.
  • CPU detection for HURD, IRIX, SCO OpenServer and (old) HPUX.
  • --ctrlc will send SIGINT to tasks running on remote computers and thus killing them.
  • --load now uses 'ps' to see immediately see the number of running processes instead of 'uptime' thus making --load react much faster.
  • Testing on Centos 3.9 and FreeBSD 7 revealed a lot of compability bugs. Some of these required quite extensive changes making this release beta quality.
  • --retries works with --onall.
  • The new --load computation now works on FreeBSD 7.
  • --nice works under tcsh.
  • GNU Parallel is officially supported in NetBSD.
  • http://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/parallel/parallel/README.html
  • GNU Parallel is accepted for openSUSE.
  • http://software.opensuse.org/package/gnu_parallel
  • GNU Parallel can be installed under Microsoft Windows (CygWin).
  • http://blogs.msdn.com/b/hpctrekker/archive/2013/03/30/preparing-and-uploading-datasets-for-hdinsight.aspx
  • Job advert that requires GNU Parallel competence.
  • http://versium.com/about/careers/
  • Parallelizing Batch Jobs for Fun and Profit.
  • http://mikeseidle.com/tech/programming/2013/03/parallelizing-batch-jobs
  • Processing Transcription Start Sites(TSS) for the entire Mouse genome.
  • http://qbrc.swmed.edu/2013/03/gnu-parallel-speeding-up-unix-commands-and-scripts/
  • GNU parallel is used throughout Scrimer
  • http://scrimer.readthedocs.org/en/latest/
  • GNU Parallel helped making public documents searchable.
  • http://danpalmer.me/blog/articles/2013-04-06-unlocking-hillsborough.html
  • Identifying big movie files (German).
  • http://blackappsolutions.wordpress.com/2013/03/23/wenn-der-plattenplatz-knapp-wird-filme-identifizieren-die-viel-platz-belegen/
  • Bug fixes and man page updates.

New in GNU Parallel 20130122 (Jan 22, 2013)

  • --sshdelay Delay starting next ssh by secs seconds. GNU parallel will pause secs seconds after starting each ssh. secs can be less than 1 seconds.
  • Official OpenBSD port: http://ftp.openbsd.org/ports/sysutils/parallel/
  • Official DragonFlyBSD package: http://www.mirrorservice.org/sites/ftp.dragonflybsd.org/packages/amd64/DragonFly-3.0.0/stable/parallel/
  • Post about niceload in Hungarian: http://commandline.blog.hu/2013/01/02/niceload
  • Bug fixes and man page updates.

New in GNU Parallel 20121222 (Jan 3, 2013)

  • --shebang-wrap makes is possible to parallelize scripts by putting parallel in the shebang line (the first #! line in scripts).
  • --delay puts in a delay after starting each job. Useful to avoid raceconditions and thundering herd problems.
  • --results now uses / as separator instead of TAB and thus creates subdirs. Thanks to Drew Frank.
  • parallel: OpenBSD CPU detection. Thanks to Michał Markowski.
  • niceload --sensor allows for negative numbers to signify a lower number means the limit is lower.
  • niceload --prg suspend a named program that is already running.
  • niceload multiple --pid suspend process ids that are already running.
  • Convertir des images avec find et imagemagick http://roger.b.over-blog.com/article-convertir-des-images-avec-find-et-imagemagick-113416637.html
  • Bug fixes and man page updates.

New in GNU Parallel 20120322 (Mar 23, 2012)

  • Parallel Process Database Dumps.
  • Using GNU Parallel to process images from Mars.
  • Using GNU Parallel with bzgrep.
  • Bug fixes and man page updates.

New in GNU Parallel 20111022 (Oct 24, 2011)

  • --tag prepends lines with the argument(s). If used with --(n)onall
  • the line will be prepended with the sshlogin instead.
  • --shellquote does not run the command but quotes it using \'s.
  • Useful for making quoted composed commands.
  • --profile can now be repeated merging multiple profiles.
  • --bibtex now gives the BibTeX entry.
  • Thanks to Aleksandr Levchuk for testing.
  • Makefile for simple .deb package.
  • cd packager/debian; make
  • sql: --list-databases lists the databases.
  • GNU Parallel officially in Fedora 16
  • http://www.spinics.net/lists/fedora-package-announce/msg67130.html
  • Blog post on recompression FLAC audio.
  • http://blog.oxplot.com/2011/08/reflac-flac-cleanser.html
  • The most complex use of ::: I a seen so far.
  • http://agentzlerich.blogspot.com/2011/09/following-up-on-previous-failures-ive.html
  • Example of using GNU Parallel with sudo.
  • http://loopkid.net/articles/2011/10/07/recursive-chmod-and-chown-on-files
  • Use wget and gnu parallel to download mp3s at rocket speed.
  • http://sdbrain.posterous.com/use-wget-and-gnu-parallel-to-download-mp3s-at
  • First CV referring to GNU Parallel.
  • http://www.peopleperhour.com/freelancers/stephen_/internet_researcher_and_protocol_develop/140602
  • Cartwright labs usage of GNU Parallel.
  • http://cartwrightlab.wikispaces.com/The+Most+Bang+for+Your+Buck
  • Bug fixes and man page updates.