Parallel - Revision history

Goldbolt: /* Install Parallel */

2019-07-20T14:43:26Z

‎Install Parallel

2019-07-20T14:43:03Z

2019-07-18T15:55:53Z

‎Using Parallel

2019-07-18T14:49:21Z

‎Using Parallel

2019-04-16T14:53:24Z

‎Using Parallel

2019-04-16T14:27:07Z

‎Using Parallel

2019-04-11T18:05:22Z

‎Using Parallel

2019-04-11T17:49:43Z

‎Using Parallel

2019-01-17T15:14:53Z

‎Using Parallel

2019-01-17T15:03:42Z

‎Using Parallel

@@ Line 29: / Line 29: @@
 =Install Parallel=
-Parallel is usually within each OS repository.
+Parallel is usually within each OS repository. There's usually a workaround if it isn't.
 Install Parallel to CentOS 6:

← Older revision		Revision as of 14:43, 20 July 2019
Line 27:		Line 27:

	find /DIR -type f \| parallel -k -j8 -n 1000 -m grep -H -n -ril STRING {}		find /DIR -type f \| parallel -k -j8 -n 1000 -m grep -H -n -ril STRING {}
		+
		+	=Install Parallel=
		+	Parallel is usually within each OS repository.
		+
		+	Install Parallel to CentOS 6:
		+	cd /etc/yum.repos.d/
		+	wget http://download.opensuse.org/repositories/home:tange/CentOS_CentOS-6/home:tange.repo
		+	yum install parallel

@@ Line 24: / Line 24: @@
      read -ep "What is the directory to delete stuff in? " delete; pwd=$(pwd); cd $delete && find . -name "*" | parallel -v -j8 rm -rf {}; cd $pwd
-Recursively grep for STRING within /DIR using 8 cores in parallel:
+Recursively grep for STRING within /DIR using 8 threads in parallel:
      find /DIR -type f | parallel -k -j8 -n 1000 -m grep -H -n -ril STRING {}

← Older revision		Revision as of 14:49, 18 July 2019
Line 23:		Line 23:

	read -ep "What is the directory to delete stuff in? " delete; pwd=$(pwd); cd $delete && find . -name "*" \| parallel -v -j8 rm -rf {}; cd $pwd		read -ep "What is the directory to delete stuff in? " delete; pwd=$(pwd); cd $delete && find . -name "*" \| parallel -v -j8 rm -rf {}; cd $pwd
		+
		+	Recursively grep for STRING within /DIR using 8 cores in parallel:
		+
		+	find /DIR -type f \| parallel -k -j8 -n 1000 -m grep -H -n -ril STRING {}

← Older revision		Revision as of 14:53, 16 April 2019
Line 22:		Line 22:
	If rm runs into "too many arguments", this'll work instead.		If rm runs into "too many arguments", this'll work instead.

−	read -ep "What is the directory to delete stuff in? " delete; pwd=$(pwd); cd $delete && find . ~~-type f~~ -name "*" \| parallel -v -j8 rm -rf {}; cd $pwd	+	read -ep "What is the directory to delete stuff in? " delete; pwd=$(pwd); cd $delete && find . -name "*" \| parallel -v -j8 rm -rf {}; cd $pwd

← Older revision		Revision as of 14:27, 16 April 2019
Line 15:		Line 15:

	read -ep "What is the directory from? " from; read -ep "What is the directory to? " to; pwd=$(pwd); cd $from && ls \| parallel -v -j8 rsync -azvP --progress {} $to{}; cd $pwd		read -ep "What is the directory from? " from; read -ep "What is the directory to? " to; pwd=$(pwd); cd $from && ls \| parallel -v -j8 rsync -azvP --progress {} $to{}; cd $pwd
		+
		+	This will <strong>delete</strong> all files within the specified directory using rm.
		+
		+	read -ep "What is the directory to delete stuff in? " delete; pwd=$(pwd); cd $delete && ls \| grep -v "./" \| grep -v "../" \| parallel -v -j8 rm -rf {}; cd $pwd
		+
		+	If rm runs into "too many arguments", this'll work instead.
		+
		+	read -ep "What is the directory to delete stuff in? " delete; pwd=$(pwd); cd $delete && find . -type f -name "*" \| parallel -v -j8 rm -rf {}; cd $pwd

← Older revision		Revision as of 18:05, 11 April 2019
Line 14:		Line 14:
	This automates the above process and just asks you where you want to copy from and then copy to. Be aware that threads are still set to 8 (per the flag "-j8") but you can change this manually.		This automates the above process and just asks you where you want to copy from and then copy to. Be aware that threads are still set to 8 (per the flag "-j8") but you can change this manually.

−	read -ep "What is the directory from? " from; read -ep "What is the directory to? " to; cd $from && ls \| parallel -v -j8 rsync -azvP --progress {} $to{}	+	read -ep "What is the directory from? " from; read -ep "What is the directory to? " to; pwd=$(pwd); cd $from && ls \| parallel -v -j8 rsync -azvP --progress {} $to{}; cd $pwd

← Older revision		Revision as of 17:49, 11 April 2019
Line 11:		Line 11:

	cd /srv/mail/ && ls \| parallel -v -j8 rsync -azvP --progress {} user@myserver.com:/srv/mail/{}		cd /srv/mail/ && ls \| parallel -v -j8 rsync -azvP --progress {} user@myserver.com:/srv/mail/{}
		+
		+	This automates the above process and just asks you where you want to copy from and then copy to. Be aware that threads are still set to 8 (per the flag "-j8") but you can change this manually.
		+
		+	read -ep "What is the directory from? " from; read -ep "What is the directory to? " to; cd $from && ls \| parallel -v -j8 rsync -azvP --progress {} $to{}

@@ Line 8: / Line 8: @@
 =Using Parallel=
-This lists the contents of /srv/mail dir and pipes it to parallel in order to rsync multiple items, 8 at a time per the -j flag, to another location. You may have to be cd'd in the /srv/mail dir first.
+This lists the contents of /srv/mail dir and pipes it to parallel in order to rsync multiple items, 8 at a time per the -j flag, to another location.
-     ls /srv/mail | parallel -v -j8 rsync -azvP --progress {} user@myserver.com:/srv/mail/{}
+     cd /srv/mail/ && ls | parallel -v -j8 rsync -azvP --progress {} user@myserver.com:/srv/mail/{}