sshoogr for your infrastructure
TRANSCRIPT
Andrey AdamovichJava/Groovy developer
DevOps guy, automation junkie
Coauthor of Groovy 2 Cookbook
Coorganizer of @latcraft and @devternity
Coach at @devchampions
Twitter: @codingandrey
••••••
03
BackgroundBig projects built by Ant, Maven, and eventually Gradle
Teams composed mostly of Java developers
Complex (sometimes, overengineered) architectures
Many environments (DEV, TEST, QA, SIT, UAT, PREPROD, PROD)
to support
••••
05
Problems IInfrastructure is influenced by (relatively) frequent architecture
changes (components, versions, layers)
We want our environments to be the same (or at least quite similar) to
avoid any side effects during development, testing and production
We don't want to spend hours/days/weeks on configuring each and
every new server and keeping them insync
•
•
•
06
Problems IIOperations guys are not always available (e.g. busy supporting
production systems or just not skilled enough)
Development infrastructure (Jenkins, Sonar, Version Control, Load
Testing etc.) also needs maintenance
We want to reuse experience available in our team and avoid throwing
in too many various trendy technologies that will fail our expectations
•
•
•
07
Ant + Gradleant.taskdef(
name: 'scp',
classname: 'o.a.t.a.t.o.ssh.Scp',
classpath: configurations.secureShell.asPath)
ant.taskdef(
name: 'sshexec',
classname: 'o.a.t.a.t.o.ssh.SSHExec',
classpath: configurations.secureShell.asPath)
01.
02.
03.
04.
05.06.
07.
08.
09.
09
Simple callant.sshexec(
host: host,
username: user,
password: password,
command: command,
trust: 'true',
failonerror: failOnError)
01.
02.
03.
04.
05.
06.
07.
10
Sshoogr featuresGroovybased SSH DSL for:
Remote command execution
File uploading/downloading
Tunneling
•••
12
Sshoogr usage (import)@Grab(
group='com.aestasit.infrastructure.sshoogr',
module='sshoogr',
version='0.9.23')
import static com.aestasit.ssh.DefaultSsh.*
01.
02.
03.
04.
05.
14
Sshoogr usage (defaults)defaultUser = 'root'
defaultKeyFile = new File('secret.pem')
execOptions {
verbose = true
showCommand = true
}
01.
02.
03.
04.
05.
06.
15
Sshoogr usage (connection)remoteSession {
url = 'user2:654321@localhost:2222'
exec 'rm ‐rf /tmp/*'
exec 'touch /var/lock/my.pid'
remoteFile('/var/my.conf').text = "enabled=true"
}
01.
02.
03.
04.
05.
06.
16
Sshoogr usage (multiline content)remoteFile('/etc/yum.repos.d/puppet.repo').text = '''
[puppet]
name=Puppet Labs Packages
baseurl=http://yum.puppetlabs.com/el/
enabled=0
gpgcheck=0
'''
01.
02.
03.
04.
05.
06.
07.
17
Sshoogr usage (appendable)remoteFile('/etc/motd') << 'Additional message'
new File('localFile') << remoteFile('/etc/motd')
localFile << remoteFile('/etc/motd') << 'msg'
01.
02.
03.
18
Sshoogr usage (file copying)remoteSession {
scp {
from { localDir "$buildDir/application" }
into { remoteDir '/var/bea/domain/application' }
}
}
01.
02.
03.
04.
05.
06.
19
Sshoogr usage (file copying)remoteSession {
scp {
from { remoteDir '/var/bea/domain/application' }
into { localDir "$buildDir/application" }
}
}
01.
02.
03.
04.
05.
06.
20
Sshoogr usage (file copying)remoteSession {
scp {
from {
remoteFile '/etc/init.d/service1'
remoteFile '/etc/init.d/service2'
}
into { localDir "$buildDir/application" }
}
}
01.
02.
03.
04.
05.
06.
07.
08.
09. 21
Sshoogr usage (command result)def result = exec(command: '/usr/bin/mycmd',
failOnError: false, showOutput: false)
if (result.exitStatus == 1) {
result.output.eachLine { line ‐>
if (line.contains('WARNING')) {
throw new RuntimeException("Warning!!!")
}
}
}
01.
02.
03.
04.
05.
06.
07.
08.
09. 22
Sshoogr usage (shortcuts)if (ok('/usr/bin/mycmd')) {
...
}
if (fail('/usr/bin/othercmd')) {
...
}
01.
02.
03.
04.
05.
06.
23
Sshoogr usage (tunnels)tunnel('1.2.3.4', 8080) { int localPort ‐>
def url = "http://localhost:${localPort}/flushCache"
def result = new URL(url).text
if (result == 'OK') {
println "Cache is flushed!"
} else {
throw new RuntimeException(result)
}
}
01.
02.
03.
04.
05.
06.
07.
08.
09. 24
Sshoogr usage (prefix/suffix)prefix('sudo ') {
exec 'rm ‐rf /var/log/abc.log'
exec 'service abc restart'
}
suffix(' >> output.log') {
exec 'yum ‐y install nginx'
exec 'yum ‐y install mc'
exec 'yum ‐y install links'
}
01.
02.
03.
04.
05.
06.
07.
08.
09. 25
Sshoogr: *.sshoogrNo need to install Groovy
Can ommit @Grab
Can ommit connection details
Scripts are more portable
Ansible for Groovy!
•••••
28
sdkman.iocurl ‐s http://get.sdkman.io | bash
sdk install sshoogr 0.9.23
sdk default sshoogr 0.9.23
sdk list sshoogr
01.
02.
03.
04.
31
SSHD MockMockSshServer.with {
command('^ls.*$') { inp, out, err, callback, env ‐>
out << '''total 20
drwxr‐xr‐x 3 1100 1100 4096 Aug 7 16:52 .
drwxr‐xr‐x 8 1100 1100 4096 Aug 1 17:53 ..
drwxr‐xr‐x 3 1100 1100 4096 Aug 7 16:49 examples
callback.onExit(0)
}
...
01.
02.
03.
04.
05.
06.
07.
08.
09. 37
SSHD Mock ...
command('^whoami.*$') { inp, out, err, callback, env ‐>
out << "root\n"
callback.onExit(0)
}
command('^du.*$') { inp, out, err, callback, env ‐>
out << "100\n"
callback.onExit(0)
}
...
01.
02.
03.
04.
05.
06.
07.
08.
09.
10.38
Gradle integrationbuildscript {
repositories { mavenCentral() }
dependencies {
classpath 'com.a....sshoogr:sshoogr‐gradle:0.9.18'
}
}
apply plugin: 'secureShell'
01.
02.
03.
04.
05.
06.
07.
42
Gradle integrationtask remoteTask << {
remoteSession("user:password@localhost:22") {
exec 'rm ‐rf /tmp/cache/'
scp "$buildDir/cache.content",
'/tmp/cache/cache.content'
}
}
01.
02.
03.
04.
05.
06.
07.
43
Groowin@Grab('com.aestasit.infrastructure.groowin:groowin:0.1.8')
import static com.aestasit.winrm.DefaultWinRM.*
01.
02.
45
GroowinremoteManagement {
host = '127.0.0.1'
user = 'Administrator'
password = 'secret'
exec 'del', 'C:\\temp.txt'
remoteFile('C:\\my.conf').text = "enabled=true"
}
01.
02.
03.
04.
05.
06.
07.
46
PUnitSimple testing tool for verifying remote server state
Uses Sshoogr and JUnit
Reuse reporting features of JUnit
As simple as ...
••••
48
PUnit: example (derby)class DerbyInstallTest
extends BasePuppetIntegrationTest {
@Before
void installDerby() {
apply("include derby")
}
...
}
01.
02.
03.
04.
05.
06.
07.
08.
49
PUnit: example (derby)@Test
void ensureDerbyRunning() {
command('service derby status > derbystatus.log')
assertTrue fileText("/root/derbystatus.log")
.contains('Derby')
assertTrue fileText("/root/derbystatus.log")
.contains('is running.')
}
01.
02.
03.
04.
05.
06.
07.
08.
50
PUnit: example (derby)@Test
void ensureCanConnect() {
Thread.sleep(10000)
uploadScript()
command('/opt/derby/db‐derby‐10.9.1.0‐bin/bin/ij ' +
'testDataScript.sql > derbytest.log')
...
01.
02.
03.
04.
05.
06.
07.
51
PUnit: example (derby) ...
// Check if the log of the insert
// operation contains the word ERROR.
assertFalse(
"The script should return at least one error",
fileText("/root/derbytest.log")
.contains('ERROR')
)
...
01.
02.
03.
04.
05.
06.
07.
08.
09. 52
PUnit: example (derby) ...
// Check on data that was inserted into a table.
assertTrue(
"The log should contain a SELECT result",
fileText("/root/derbytest.log")
.contains('Grand Ave.')
)
}
01.
02.
03.
04.
05.
06.
07.
08.
53
PUnit: example (jenkins)session {
tunnel ('127.0.0.1', 8080) { int localPort ‐>
def driver = new HtmlUnitDriver(false)
driver.manage()
.timeouts()
.pageLoadTimeout(300, TimeUnit.SECONDS)
.implicitlyWait(30, TimeUnit.SECONDS)
driver.get("http://127.0.0.1:${localPort}/login")
...
01.
02.
03.
04.
05.
06.
07.
08.
09.
10.54
PUnit: example (jenkins) ...
def input = driver.findElement(By.name('j_username'))
input.sendKeys('john')
input = driver.findElement(By.name('j_password'))
input.sendKeys('123456')
input.submit()
...
01.
02.
03.
04.
05.
06.
07.
55
PUnit: example (jenkins) ...
def wait = new WebDriverWait(driver, 30)
wait.until ExpectedConditions.
presenceOfElementLocated (By.linkText('John Doe'))
...
}
}
01.
02.
03.
04.
05.
06.
07.
56
PUnit: example (svn)session {
tunnel ('127.0.0.1', 80) { int localPort ‐>
// Initilize repository connection data.
DAVRepositoryFactory.setup()
def url = SVNURL.create('http', null, '127.0.0.1',
localPort, 'repos/cafebabe', true)
def repository = SVNRepositoryFactory.create(url)
println "Verifying SVN repository at ${url}"
...
01.
02.
03.
04.
05.
06.
07.
08.
09. 57
PUnit: example (svn) ...
// Setup credentials.
def authManager = SVNWCUtil.
createDefaultAuthenticationManager('joe', '123456')
repository.setAuthenticationManager(authManager)
// Verify repository is at revision 0.
assertEquals 0, repository.getLatestRevision()
...
01.
02.
03.
04.
05.
06.
07.
08.
09. 58
PUnit: example (svn) ...
// Commit first revision.
ISVNEditor editor = repository.
getCommitEditor("Initial commit.", null)
editor.with {
openRoot(‐1)
addFile('dummy.txt', null, ‐1)
applyTextDelta('dummy.txt', null)
def deltaGenerator = new SVNDeltaGenerator()
01.
02.
03.
04.
05.
06.
07.
08.
09. 59
PUnit: example (svn) ...
def checksum = deltaGenerator.sendDelta('dummy.txt',
new ByteArrayInputStream("data".getBytes()),
editor, true)
closeFile('dummy.txt', checksum)
def commitInfo = closeEdit()
println commitInfo
}
...
01.
02.
03.
04.
05.
06.
07.
08.
09. 60
PUnit: example (svn) ...
// Verify repository is at revision 1 now.
assertEquals 1, repository.getLatestRevision()
}
}
01.
02.
03.
04.
05.
61
Sshoogr is...Battletested Groovy DSL for SSH connectivity
Executable and portable scripting tool
Easily integratable with any Java/Groovy library
•••
65
Sshoogr can be used for...Provisioning your servers and IoT devices
Executing remote orchestration commands
Testing and monitoring infrastructure state
•••
66
Next stepsResource definitions
Command rollbacks
Parallel execution
XSS utilities
Extend integration tests
Better documentation
••••••
67
Source codeSshoogr: https://github.com/aestasit/sshoogr
Sshoogr Gradle: https://github.com/aestasit/sshoogrgradle
Groowin: https://github.com/aestasit/groowin
Groowin Gradle: https://github.com/aestasit/groowingradle
••••
69