dtools documentation
TRANSCRIPT
-
8/8/2019 dtools documentation
1/23
dtools DocumentationRelease 4.0
Andrs J. Daz
November 17, 2010
-
8/8/2019 dtools documentation
2/23
-
8/8/2019 dtools documentation
3/23
CONTENTS
1 Installation 3
1.1 Gentoo & Exherbo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 First Contact 52.1 Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3 The dtools database 7
3.1 Manage the database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4 Patterns 9
4.1 System pattern (sys) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.2 Output pattern (out) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.3 Expression pattern (exp) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.4 Tag pattern (tag) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.5 Combining patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
5 Commands 11
5.1 The add command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5.2 The ipmitool command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5.3 The list command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
5.4 The ping command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
5.5 The rscp command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
5.6 The scp command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
5.7 The script command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
5.8 The ssh command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.9 The ssh-copy-id command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.10 The sudo command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.11 The tag command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
6 Examples 15
6.1 Add a number of hosts manually . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6.2 Bulk adding hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6.3 Authorize hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6.4 Ping all hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
7 For impatients 17
8 License 19
i
-
8/8/2019 dtools documentation
4/23
-
8/8/2019 dtools documentation
5/23
dtools Documentation, Release 4.0
Author Andres J. Diaz
Abstract dtools is a suite of utilities written in pure bash to handle a number of nodes in a distributed
network, its similar to other distributed shells, like dsh or pyDSH, but designed to be modular
and easy-to-extend.
CONTENTS 1
mailto:[email protected]://www.netfort.gr.jp/~dancer/software/dsh.html.enhttp://pydsh.sourceforge.net/http://pydsh.sourceforge.net/http://www.netfort.gr.jp/~dancer/software/dsh.html.enmailto:[email protected] -
8/8/2019 dtools documentation
6/23
dtools Documentation, Release 4.0
2 CONTENTS
-
8/8/2019 dtools documentation
7/23
CHAPTER
ONE
INSTALLATION
Here are the installation instructions for dtools, our goal is to provide a custom installation for each GNU/Linux
and UNIX distributions, in the meanwhile, a generic overview is described.
First you can download the currently working branch from github main repository using git protocol:
$ git clone git://github.com/ajdiaz/dtools
Or even download an specified version:
$ wget -O dtools-4.0.tar.gz http://github.com/ajdiaz/dtools/tarball/4.0
Then, once unpacked, you can run dtools directly from the working directory, prepending the DTOOLS_LIB
variable to change the library path:
$ cd dtools-4.0
$ DTOOLS_LIB=lib/ ./dt -h
Or you can copy these files to the system as root:
# cp dt dthost dtstatus /usr/bin
# cp -r lib /usr/lib/dtools
# cp -r doc/man/\*.1 /usr/share/man/man1
Also you have packages for a number of GNU/Linux distributions
1.1 Gentoo & Exherbo
Please use the paludis repository ajdiaz-exheres, and then:
# paludis -i dtools
Note: If you like to contribute with any package, please contact me to devel atlists.connectical.com
3
http://github.com/ajdiaz/ajdiaz-exhereshttp://github.com/ajdiaz/ajdiaz-exheres -
8/8/2019 dtools documentation
8/23
dtools Documentation, Release 4.0
4 Chapter 1. Installation
-
8/8/2019 dtools documentation
9/23
CHAPTER
TWO
FIRST CONTACT
This is the main syntax for dt invokation:
dt [options] : [args]
The options can modify the normal behaviour of dt, including the dtools database location, the number of paralell
executions and more.
The pattern is a way to get the targets. The targets are the nodes in database where a command will be executed.
Usually the pattern get a subset of nodes in database.
The command is the action which will be executed in the remote nodes affected by the pattern matching. Note
that the could be not just a shell command, but a dtools command, for example the ssh command run using ssh
protocol a number of commands which will be passed in args.
Note: Please read the section Examples for a number of well explained examples, which is the more usefull part
of this documentation too.
2.1 OptionsThe dtools script accept the following options:
-h X --help X display help about X command or this help screen.
-D X --db use X as known_hosts database.
-p --pretend pretend only and do nothing.
-C X --coproc X use X coprocs (default:5).
-T X --timeout X use X seconds of timeout (default:5).
-i --interactive interactive, an alias to -C 0
-N --no-color supress color output
2.2 Features
Modular, so its easy to extend, just only need to create your own pattern or command file.
Thinking in sysadmin. No development skills requirement to extend the software.
Fast. Using coprocs to run a number of hosts in paralel.
Easy to parse. The output format is a colon-separated list, which is easy to parse for non-humans, but
easy-to-read for humans too.
Easy to deploy. Written in bash, just only need a modern shell to run it.
Tested. Used in production environment for years, with more of 400 hosts.
5
-
8/8/2019 dtools documentation
10/23
dtools Documentation, Release 4.0
6 Chapter 2. First Contact
-
8/8/2019 dtools documentation
11/23
-
8/8/2019 dtools documentation
12/23
dtools Documentation, Release 4.0
8 Chapter 3. The dtools database
-
8/8/2019 dtools documentation
13/23
CHAPTER
FOUR
PATTERNS
As we explained in Introduction, the patterns are a way to filter the dtools database to find a properly target set.
You can create your own patterns as described in Developmentsection, but dtools has a number of them for you.
4.1 System pattern (sys)
Description Acts over hostname passed in pattern syntax.
Syntax sys:
Example sys:host1.connectical.com
4.2 Output pattern (out)
Description Get the database from the stdin. Usefull to combine with other patterns (see Combin-
ing patterns section for more information).Syntax out:
Example out::
4.3 Expression pattern (exp)
Description Filter the database using a regular expression matching for host.
Syntax exp:
Example exp:db.*
4.4 Tag pattern (tag)
Description Get from the database the hosts which have the specified tag operation.
Syntax tag:
Example tag:database
OR Operation
Operator ,
Meaning return hosts which match which almost one of two elements separated by the operator.Examples tag:database,webserver
9
-
8/8/2019 dtools documentation
14/23
dtools Documentation, Release 4.0
AND Operation
Operator +
Meaning return hosts which match with both tags.
Examples tag:database+oracle
EXCEPT Operation
Operator -
Meaning returng hosts which match for the first tag, but not for the second one.
Examples tag:database-oracle
4.5 Combining patterns
You can filter the result of one pattern with other pattern. To do this just append two-colons and the next pattern
to filter. For example, lets suppose that you can filter your database using a tag pattern, i.e. all nodes with tag
mysql, and then you can filter this result with nodes which name starts with db, then you can do something similarto this:
tag:mysql:exp:db.*
The pattern combination are a powerfull tool to play with database filters, neverthless, in practice, one pattern
would be enough for every day use.
An usefull case of pattern combination is the out pattern, with the out pattern you can read hosts from stdin
and filter them with normal patterns, just in a combination form. For example:
out:tag:mysql
10 Chapter 4. Patterns
-
8/8/2019 dtools documentation
15/23
CHAPTER
FIVE
COMMANDS
The commands are the actions that dtools can performs. Be carefull with commands, when we talk about dtools
commands we are not talking about running commands on remote systems, in fact, run a remote command using
SSH is just only dtool command. To avoid this problem we callactions to the dtools commands, leaving commands
for the remote commands than we perform over SSH or other protocol.
There are a list of currently available commands in dtool:
add add host to dtools database
ipmitool manage IPMI interface on remote hosts
list list hosts
ping ping hosts
rscp reverse scp from host to local machine
scp classic scp copy to hosts
script execute a local script in remote hosts
ssh execute a command in remote hosts
ssh-copy-id distribute a public key in hosts
sudo execute a command in remote hosts with privilegies
tag manage tagging features
5.1 The add command
Syntax add [tags]+
Meaning add the affected hosts to the host database, tagging them if any tag is supplied.
Context local
Requirements None
Example dt sys:mynewhost add
5.2 The ipmitool command
Syntax ipmitool [ipmitool_options]
Meaning perform IPMI actions to the affected hosts using ipmitool
Context remote
Requirements ipmitool
Example dt tag:all ipmitool cycle
11
-
8/8/2019 dtools documentation
16/23
dtools Documentation, Release 4.0
5.3 The list command
Syntax list
Meaning list affected hosts
Context local
Requirements None
Example dt tag:all list
5.4 The ping command
Syntax ping
Meaning perform a classic ICMP ping to the affected host, and measure the trip time.
Context remote
Requirements ping
Example dt tag:all ping
5.5 The rscp command
Syntax rscp
Meaning reverse copy (from remote host to local host), using SCP.
Context remote
Requirements scp
Example dt tag:all rscp /etc/passwd /tmp/override_passwd
5.6 The scp command
Syntax scp
Meaning direct copy (from local host to remote host) using SCP
Context remote
Requirements scp
Example dt tag:all scp /etc/passwd /etc/passwd
5.7 The script command
Syntax script
Meaning run in the remote host the script which is located in local host.
Context remote
Requirements ssh
Example dt tag:all script ./create_users.sh
12 Chapter 5. Commands
-
8/8/2019 dtools documentation
17/23
dtools Documentation, Release 4.0
5.8 The ssh command
Syntax ssh [ssh_options]
Meaning run in the remote host the arbitrary action using SSH
Context remote
Requirements ssh
Example dt tag:all ssh date
5.9 The ssh-copy-id command
Syntax ssh-copy-id
Meaning distributed a pubkey to affected hosts.
Context remote
Requirements ssh-copy-id
Example dt tag:all ssh-copy-id ~/.ssh/id_rsa.pub
5.10 The sudo command
Syntax sudo [sudo_options]
Meaning perform a remote action with superuser permissions with sudo
Context remote
Requirements ssh (local), sudo (remote)
Example dt tag:all sudo deluser testing
5.11 The tag command
Syntax tag [ +, - ]+
Meaning add or remove tags to the affected hosts.
Context local
Requirements None
Example dt tag:all tag +everyhost
5.8. The ssh command 13
-
8/8/2019 dtools documentation
18/23
dtools Documentation, Release 4.0
14 Chapter 5. Commands
-
8/8/2019 dtools documentation
19/23
CHAPTER
SIX
EXAMPLES
Here are a number of examples using dtools, from the basic use to more advanced use.
6.1 Add a number of hosts manually
$ dt sys:myhost1.com add
okay::dt:add:myhost1.com:
$ dt sys:myhost2.com add
okay::dt:add:myhost2.com:
...
6.2 Bulk adding hosts
Lets suppose that host are in a file called myhosts.txt, one host per line:
$ cat myhosts.txtmyhost1.com
myhost2.com
$ cat myhosts.txt | dt out:: add
okay::dt:add:myhost1.com:
okay::dt:add:myhost1.com:
6.3 Authorize hosts
Once hosts are added to dtools database, you might want authorize them. When a host is authorized, you can use
SSH commands without need to type passwords, using public key handshake.
$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (~/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in asd.
Your public key has been saved in asd.pub.
The key fingerprint is:
cf:19:ce:44:45:35:78:11:ce:04:23:eb:fb:e9:9f:cc ajdiaz@boole
The keys randomart image is:
+--[ RSA 2048]----+
| ..=+*o |
| +.=.. |
| o .o |
| o |
15
-
8/8/2019 dtools documentation
20/23
dtools Documentation, Release 4.0
| S + |
| * + |
| B |
| . + . |
| .+.E |
+-----------------+
$ dt exp:.* ssh-copy-id ~/.ssh/id_rsaokay::dt:ssh-copy-id:myhost1.com:
okay::dt:ssh-copy-id:myhost1.com:
6.4 Ping all hosts
$ dt exp:.* ping
okay::dt:ping:myhost1.com:0ms
okay::dt:ping:myhost1.com:0ms
16 Chapter 6. Examples
-
8/8/2019 dtools documentation
21/23
CHAPTER
SEVEN
FOR IMPATIENTS
First add hosts to database
$ dt sys:myhost.com add
$ dt sys:otherhost.net add
Then send your public key for ssh authentication
$ dt exp:.* ssh-copy-id ~/.ssh/id_dsa.pub
Finally, enjoy!
$ dt exp:.* list
17
-
8/8/2019 dtools documentation
22/23
dtools Documentation, Release 4.0
18 Chapter 7. For impatients
-
8/8/2019 dtools documentation
23/23
CHAPTER
EIGHT
LICENSE
This code is releasing under GPLv2 license.
http://www.gnu.org/licenses/gpl-2.0-standalone.htmlhttp://www.gnu.org/licenses/gpl-2.0-standalone.html