organizing the world of cq rest infinitive possibilities by arkadiusz kita
DESCRIPTION
TRANSCRIPT
Arkadiusz Kita CQ Developer
Wednesday morning, 9th April 2014...
...another beautiful day at work!
There is time for my favouritecoffee!
Wednesday morning, 9th April 2014...
...another beautiful day at work!
...but wait - there is an importante-mail in my inbox!
1 The client needs PoC ASAP
2 Next week 1 mln hits campaign!
3 Agency is ready for content entry!
Mainframe
4 Ad-hoc infrastructure!
5 Monitoring/maintenance required!
1 The client needs PoC ASAP
2 Next week 1 mln hits campaign!
3 Agency is ready for content entry!
Mainframe
4 Ad-hoc infrastructure!
5 Monitoring/maintenance required!
1 The client needs PoC ASAP
2 Next week 1 mln hits campaign!
3 Agency is ready for content entry!
Mainframe
4 Ad-hoc infrastructure!
5 Monitoring/maintenance required!
1 The client needs PoC ASAP
2 Next week 1 mln hits campaign!
3 Agency is ready for content entry!
Mainframe
4 Ad-hoc infrastructure!
5 Monitoring/maintenance required!
1 The client needs PoC ASAP
2 Next week 1 mln hits campaign!
3 Agency is ready for content entry!
Mainframe
4 Ad-hoc infrastructure!
5 Monitoring/maintenance required!
It seems like huge pressure and a lot of work to do...
...or maybe rather a lot of CQ clicks!
Managing a cloud of...
...thousands of AEM/CQ instances
...hundreds of dev, test, qa, uat, live environments...tens of client deployments
in a consistent way!
Managing a cloud of...
...thousands of AEM/CQ instances...hundreds of dev, test, qa, uat, live environments
...tens of client deployments
in a consistent way!
Managing a cloud of...
...thousands of AEM/CQ instances...hundreds of dev, test, qa, uat, live environments
...tens of client deployments
in a consistent way!
Managing a cloud of...
...thousands of AEM/CQ instances...hundreds of dev, test, qa, uat, live environments
...tens of client deployments
in a consistent way!
Providing condensed...
checklists knowledge know-how
...for developers/administrators
in a reusable way!
Providing condensed...
checklists
knowledge know-how
...for developers/administrators
in a reusable way!
Providing condensed...
checklists knowledge
know-how
...for developers/administrators
in a reusable way!
Providing condensed...
checklists knowledge know-how
...for developers/administrators
in a reusable way!
Providing condensed...
checklists knowledge know-how
...for developers/administrators
in a reusable way!
Last but not least!
1 Automating repetetive tasks
2 Improving safety and security by not touching anything twice3 Focusing on real challenges!!! Discovering new possibilities!4 Being happy, not bored!5 Loving what you do!
Last but not least!
1 Automating repetetive tasks2 Improving safety and security by not touching anything twice
3 Focusing on real challenges!!! Discovering new possibilities!4 Being happy, not bored!5 Loving what you do!
Last but not least!
1 Automating repetetive tasks2 Improving safety and security by not touching anything twice3 Focusing on real challenges!!! Discovering new possibilities!
4 Being happy, not bored!5 Loving what you do!
Last but not least!
1 Automating repetetive tasks2 Improving safety and security by not touching anything twice3 Focusing on real challenges!!! Discovering new possibilities!4 Being happy, not bored!
5 Loving what you do!
Last but not least!
1 Automating repetetive tasks2 Improving safety and security by not touching anything twice3 Focusing on real challenges!!! Discovering new possibilities!4 Being happy, not bored!5 Loving what you do!
Use AEM/CQ REST infinitive possibilities!
UNIXUNIXTry the github.com/Cognifide/CQ-Unix-Toolkit!
Start using it!Help us develop it!
What is the CQ Unix Toolkit?
+ + = UNIXUNIX
open source user-friendly curl wrapper
set of independent simple micro tools
AEM/CQ HTTP REST semi-code/semi-documentation
allows to control CQ/dispatcher by HTTP
Commons are extracted into one cqapi tool.
What is the CQ Unix Toolkit?
+ + = UNIXUNIX
open source user-friendly curl wrapper
set of independent simple micro tools
AEM/CQ HTTP REST semi-code/semi-documentation
allows to control CQ/dispatcher by HTTP
Commons are extracted into one cqapi tool.
What is the CQ Unix Toolkit?
+ + = UNIXUNIX
open source user-friendly curl wrapper
set of independent simple micro tools
AEM/CQ HTTP REST semi-code/semi-documentation
allows to control CQ/dispatcher by HTTP
Commons are extracted into one cqapi tool.
What is the CQ Unix Toolkit?
+ + = UNIXUNIX
open source user-friendly curl wrapper
set of independent simple micro tools
AEM/CQ HTTP REST semi-code/semi-documentation
allows to control CQ/dispatcher by HTTP
Commons are extracted into one cqapi tool.
What is the CQ Unix Toolkit?
+ + = UNIXUNIX
open source user-friendly curl wrapper
set of independent simple micro tools
AEM/CQ HTTP REST semi-code/semi-documentation
allows to control CQ/dispatcher by HTTP
Commons are extracted into one cqapi tool.
What is the CQ Unix Toolkit?
+ + = UNIXUNIX
open source user-friendly curl wrapper
set of independent simple micro tools
AEM/CQ HTTP REST semi-code/semi-documentation
allows to control CQ/dispatcher by HTTP
Commons are extracted into one cqapi tool.
Use cases
Case 1 — distributed REST knowledge
No common place for AEM/CQ REST documentation!
Case 2 — hacking packages
1 Installing hotfixes/releases automaticallycqput; cqrun
2 Synchronizing content between instancescqpkg; cqput; cqbld; cqcp; cqrun; cqsnp
3 Making path-based backupscqpkg; cqput; cqbld; cqcp
4 Uninstalling deleted packagescqsnp
5 Clearing content pathscqpkg; cqput; cqrun
6 Moving paths from repository externallycqpkg; cqput; cqrun; cqsnp; cqsnp
7 Deleting unknown/garbage packagescqls; cqdel; cqsnp
Case 2 — hacking packages
1 Installing hotfixes/releases automaticallycqput; cqrun
2 Synchronizing content between instancescqpkg; cqput; cqbld; cqcp; cqrun; cqsnp
3 Making path-based backupscqpkg; cqput; cqbld; cqcp
4 Uninstalling deleted packagescqsnp
5 Clearing content pathscqpkg; cqput; cqrun
6 Moving paths from repository externallycqpkg; cqput; cqrun; cqsnp; cqsnp
7 Deleting unknown/garbage packagescqls; cqdel; cqsnp
Case 2 — hacking packages
1 Installing hotfixes/releases automaticallycqput; cqrun
2 Synchronizing content between instancescqpkg; cqput; cqbld; cqcp; cqrun; cqsnp
3 Making path-based backupscqpkg; cqput; cqbld; cqcp
4 Uninstalling deleted packagescqsnp
5 Clearing content pathscqpkg; cqput; cqrun
6 Moving paths from repository externallycqpkg; cqput; cqrun; cqsnp; cqsnp
7 Deleting unknown/garbage packagescqls; cqdel; cqsnp
Case 2 — hacking packages
1 Installing hotfixes/releases automaticallycqput; cqrun
2 Synchronizing content between instancescqpkg; cqput; cqbld; cqcp; cqrun; cqsnp
3 Making path-based backupscqpkg; cqput; cqbld; cqcp
4 Uninstalling deleted packagescqsnp
5 Clearing content pathscqpkg; cqput; cqrun
6 Moving paths from repository externallycqpkg; cqput; cqrun; cqsnp; cqsnp
7 Deleting unknown/garbage packagescqls; cqdel; cqsnp
Case 2 — hacking packages
1 Installing hotfixes/releases automaticallycqput; cqrun
2 Synchronizing content between instancescqpkg; cqput; cqbld; cqcp; cqrun; cqsnp
3 Making path-based backupscqpkg; cqput; cqbld; cqcp
4 Uninstalling deleted packagescqsnp
5 Clearing content pathscqpkg; cqput; cqrun
6 Moving paths from repository externallycqpkg; cqput; cqrun; cqsnp; cqsnp
7 Deleting unknown/garbage packagescqls; cqdel; cqsnp
Case 2 — hacking packages
1 Installing hotfixes/releases automaticallycqput; cqrun
2 Synchronizing content between instancescqpkg; cqput; cqbld; cqcp; cqrun; cqsnp
3 Making path-based backupscqpkg; cqput; cqbld; cqcp
4 Uninstalling deleted packagescqsnp
5 Clearing content pathscqpkg; cqput; cqrun
6 Moving paths from repository externallycqpkg; cqput; cqrun; cqsnp; cqsnp
7 Deleting unknown/garbage packagescqls; cqdel; cqsnp
Case 2 — hacking packages
1 Installing hotfixes/releases automaticallycqput; cqrun
2 Synchronizing content between instancescqpkg; cqput; cqbld; cqcp; cqrun; cqsnp
3 Making path-based backupscqpkg; cqput; cqbld; cqcp
4 Uninstalling deleted packagescqsnp
5 Clearing content pathscqpkg; cqput; cqrun
6 Moving paths from repository externallycqpkg; cqput; cqrun; cqsnp; cqsnp
7 Deleting unknown/garbage packagescqls; cqdel; cqsnp
Case 2 — hacking packages
1 Installing hotfixes/releases automaticallycqput; cqrun
2 Synchronizing content between instancescqpkg; cqput; cqbld; cqcp; cqrun; cqsnp
3 Making path-based backupscqpkg; cqput; cqbld; cqcp
4 Uninstalling deleted packagescqsnp
5 Clearing content pathscqpkg; cqput; cqrun
6 Moving paths from repository externallycqpkg; cqput; cqrun; cqsnp; cqsnp
7 Deleting unknown/garbage packagescqls; cqdel; cqsnp
Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring2 cqwfl – running/stale workflows status3 cqchk – repository consistency checks4 cqtpm – TarPM compaction5 cqmrg – Lucene indexes merge6 cqgc – DataStore garbage collection7 cqdam – AEM/CQ DAM Health Tool8 cqosgi + diff – bundle states (by using patterns)$ cqosgi -m > pattern-file;$ cqosgi -s org.apache.sling.jcr.davexStopping bundle: org.apache.sling.jcr.davex (#109)$ cqosgi -m | diff -q - pattern-fileFiles - and pattern-file differ
Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring
2 cqwfl – running/stale workflows status3 cqchk – repository consistency checks4 cqtpm – TarPM compaction5 cqmrg – Lucene indexes merge6 cqgc – DataStore garbage collection7 cqdam – AEM/CQ DAM Health Tool8 cqosgi + diff – bundle states (by using patterns)$ cqosgi -m > pattern-file;$ cqosgi -s org.apache.sling.jcr.davexStopping bundle: org.apache.sling.jcr.davex (#109)$ cqosgi -m | diff -q - pattern-fileFiles - and pattern-file differ
Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring2 cqwfl – running/stale workflows status
3 cqchk – repository consistency checks4 cqtpm – TarPM compaction5 cqmrg – Lucene indexes merge6 cqgc – DataStore garbage collection7 cqdam – AEM/CQ DAM Health Tool8 cqosgi + diff – bundle states (by using patterns)$ cqosgi -m > pattern-file;$ cqosgi -s org.apache.sling.jcr.davexStopping bundle: org.apache.sling.jcr.davex (#109)$ cqosgi -m | diff -q - pattern-fileFiles - and pattern-file differ
Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring2 cqwfl – running/stale workflows status3 cqchk – repository consistency checks
4 cqtpm – TarPM compaction5 cqmrg – Lucene indexes merge6 cqgc – DataStore garbage collection7 cqdam – AEM/CQ DAM Health Tool8 cqosgi + diff – bundle states (by using patterns)$ cqosgi -m > pattern-file;$ cqosgi -s org.apache.sling.jcr.davexStopping bundle: org.apache.sling.jcr.davex (#109)$ cqosgi -m | diff -q - pattern-fileFiles - and pattern-file differ
Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring2 cqwfl – running/stale workflows status3 cqchk – repository consistency checks4 cqtpm – TarPM compaction
5 cqmrg – Lucene indexes merge6 cqgc – DataStore garbage collection7 cqdam – AEM/CQ DAM Health Tool8 cqosgi + diff – bundle states (by using patterns)$ cqosgi -m > pattern-file;$ cqosgi -s org.apache.sling.jcr.davexStopping bundle: org.apache.sling.jcr.davex (#109)$ cqosgi -m | diff -q - pattern-fileFiles - and pattern-file differ
Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring2 cqwfl – running/stale workflows status3 cqchk – repository consistency checks4 cqtpm – TarPM compaction5 cqmrg – Lucene indexes merge
6 cqgc – DataStore garbage collection7 cqdam – AEM/CQ DAM Health Tool8 cqosgi + diff – bundle states (by using patterns)$ cqosgi -m > pattern-file;$ cqosgi -s org.apache.sling.jcr.davexStopping bundle: org.apache.sling.jcr.davex (#109)$ cqosgi -m | diff -q - pattern-fileFiles - and pattern-file differ
Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring2 cqwfl – running/stale workflows status3 cqchk – repository consistency checks4 cqtpm – TarPM compaction5 cqmrg – Lucene indexes merge6 cqgc – DataStore garbage collection
7 cqdam – AEM/CQ DAM Health Tool8 cqosgi + diff – bundle states (by using patterns)$ cqosgi -m > pattern-file;$ cqosgi -s org.apache.sling.jcr.davexStopping bundle: org.apache.sling.jcr.davex (#109)$ cqosgi -m | diff -q - pattern-fileFiles - and pattern-file differ
Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring2 cqwfl – running/stale workflows status3 cqchk – repository consistency checks4 cqtpm – TarPM compaction5 cqmrg – Lucene indexes merge6 cqgc – DataStore garbage collection7 cqdam – AEM/CQ DAM Health Tool
8 cqosgi + diff – bundle states (by using patterns)$ cqosgi -m > pattern-file;$ cqosgi -s org.apache.sling.jcr.davexStopping bundle: org.apache.sling.jcr.davex (#109)$ cqosgi -m | diff -q - pattern-fileFiles - and pattern-file differ
Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring2 cqwfl – running/stale workflows status3 cqchk – repository consistency checks4 cqtpm – TarPM compaction5 cqmrg – Lucene indexes merge6 cqgc – DataStore garbage collection7 cqdam – AEM/CQ DAM Health Tool8 cqosgi + diff – bundle states (by using patterns)
$ cqosgi -m > pattern-file;$ cqosgi -s org.apache.sling.jcr.davexStopping bundle: org.apache.sling.jcr.davex (#109)$ cqosgi -m | diff -q - pattern-fileFiles - and pattern-file differ
Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring2 cqwfl – running/stale workflows status3 cqchk – repository consistency checks4 cqtpm – TarPM compaction5 cqmrg – Lucene indexes merge6 cqgc – DataStore garbage collection7 cqdam – AEM/CQ DAM Health Tool8 cqosgi + diff – bundle states (by using patterns)$ cqosgi -m > pattern-file;$ cqosgi -s org.apache.sling.jcr.davexStopping bundle: org.apache.sling.jcr.davex (#109)$ cqosgi -m | diff -q - pattern-fileFiles - and pattern-file differ
Case 4 — common devops tasks
1 cqosgi – automated bundle management2 cqpkg – create ad-hoc empty packages3 cqrepkg – fast offline repackaging for your content snapshots4 cqclr – clear dispatcher cache5 cqcfg – change bundle configuration remotely
configure SMTP clienttune Sling Main servlet
6 cqjcr – list JCR nodes, change and add themdynamically-scoped backups:cqjcr; cqpkg; cqbld; cqcp; cqdelchange Sling redirect/disable ContentFinder
Case 4 — common devops tasks
1 cqosgi – automated bundle management
2 cqpkg – create ad-hoc empty packages3 cqrepkg – fast offline repackaging for your content snapshots4 cqclr – clear dispatcher cache5 cqcfg – change bundle configuration remotely
configure SMTP clienttune Sling Main servlet
6 cqjcr – list JCR nodes, change and add themdynamically-scoped backups:cqjcr; cqpkg; cqbld; cqcp; cqdelchange Sling redirect/disable ContentFinder
Case 4 — common devops tasks
1 cqosgi – automated bundle management2 cqpkg – create ad-hoc empty packages
3 cqrepkg – fast offline repackaging for your content snapshots4 cqclr – clear dispatcher cache5 cqcfg – change bundle configuration remotely
configure SMTP clienttune Sling Main servlet
6 cqjcr – list JCR nodes, change and add themdynamically-scoped backups:cqjcr; cqpkg; cqbld; cqcp; cqdelchange Sling redirect/disable ContentFinder
Case 4 — common devops tasks
1 cqosgi – automated bundle management2 cqpkg – create ad-hoc empty packages3 cqrepkg – fast offline repackaging for your content snapshots
4 cqclr – clear dispatcher cache5 cqcfg – change bundle configuration remotely
configure SMTP clienttune Sling Main servlet
6 cqjcr – list JCR nodes, change and add themdynamically-scoped backups:cqjcr; cqpkg; cqbld; cqcp; cqdelchange Sling redirect/disable ContentFinder
Case 4 — common devops tasks
1 cqosgi – automated bundle management2 cqpkg – create ad-hoc empty packages3 cqrepkg – fast offline repackaging for your content snapshots4 cqclr – clear dispatcher cache
5 cqcfg – change bundle configuration remotelyconfigure SMTP clienttune Sling Main servlet
6 cqjcr – list JCR nodes, change and add themdynamically-scoped backups:cqjcr; cqpkg; cqbld; cqcp; cqdelchange Sling redirect/disable ContentFinder
Case 4 — common devops tasks
1 cqosgi – automated bundle management2 cqpkg – create ad-hoc empty packages3 cqrepkg – fast offline repackaging for your content snapshots4 cqclr – clear dispatcher cache5 cqcfg – change bundle configuration remotely
configure SMTP clienttune Sling Main servlet
6 cqjcr – list JCR nodes, change and add themdynamically-scoped backups:cqjcr; cqpkg; cqbld; cqcp; cqdelchange Sling redirect/disable ContentFinder
Case 4 — common devops tasks
1 cqosgi – automated bundle management2 cqpkg – create ad-hoc empty packages3 cqrepkg – fast offline repackaging for your content snapshots4 cqclr – clear dispatcher cache5 cqcfg – change bundle configuration remotely
configure SMTP clienttune Sling Main servlet
6 cqjcr – list JCR nodes, change and add themdynamically-scoped backups:cqjcr; cqpkg; cqbld; cqcp; cqdelchange Sling redirect/disable ContentFinder
Development
Create new tools with us!Organize the AEM/CQ REST World!
DevelopmentCreate new tools with us!
Organize the AEM/CQ REST World!
Rule 1 — Capture day to day requests
Rule 2 — Write user-friendly usage messages
$ cqbldUsage: cqbld [OPTION...] package-nameBuild (rebuild) already uploaded package by group id and name in CQ PackageManager using instance URL. Examples: cqbld -u admin pack # Build package named pack cqbld -u admin -g GRP pack # Build package named pack in group GRP cqbld -i http://localhost:5510 # Build package for localhost instance on tcp -g com.group stuff # port 5510 named stuff in group:com.group -p secret # with password provided: secret
Options:
-u use specified usernamed for connection -p use provided password for authentication -i use specified instance URL to connect -g locate package by additional group ID
Rule 3 — Choose descriptive code over comments
# Invoke curl tool with current referer and # instance URL and credentials to get list # of packages STATUS=`$CURLBIN -s --write-out "%{http_code}"\ -u "$u:$p" -H "Referer: $url/crx/packmgr"\ -F cmd=ls "$url/crx/packmgr/service.jsp"`
Rule 3 — Choose descriptive code over comments
# Invoke curl tool with current referer and # instance URL and credentials to get list # of packages STATUS=`$CURLBIN -s --write-out "%{http_code}"\ -u "$u:$p" -H "Referer: $url/crx/packmgr"\ -F cmd=ls "$url/crx/packmgr/service.jsp"`
Rule 3 — Choose descriptive code over comments
PACKAGE_MANAGER_PATH="/crx/packmgr"REFERER="${instance}${PACKAGE_MANAGER_PATH}" AUTH="${username}:${password}" FILEPATH="${PACKAGE_MANAGER_PATH}/service.jsp" FULL_URL="${instance}${FILEPATH}" REFERER_HEADER= "Referer: ${REFERER}" COMMAND="cmd=ls"
Use shellcheck.net to validate POSIX shell syntax!
Rule 3 — Choose descriptive code over comments
STATUS=$("${CURLBIN}" \ -s \ --write-out "%{http_code}" \ -u "${AUTH}" \ -H "${REFERER_HEADER}" \ -F "${COMMAND}" \ "${FULL_URL}")
Rule 3 — Choose descriptive code over comments
STATUS=$("${CURLBIN}" \ -s \ --write-out "%{http_code}" \ -u "${AUTH}" \ -H "${REFERER_HEADER}" \ -F "${COMMAND}" \ "${FULL_URL}")
Rule 4 — Use POSIX getopts for custom parameters
while getopts ":abc${apigetopts}" optdo case "${opt}" in
...
\?) echo "Invalid option: -${OPTARG}" >&2; _usage;; :) echo "Option -${OPTARG} requires an argument." >&2;
_usage;; esac done
Rule 5 — Use cqapi to detect/handle curl command
cqapi
shared code, status, parsing common options, error handling
Rule 6 — Use the most usable REST HTTP
pick!
rate!
choose!
Rule 7 — Reserve API cmdline options for CQ/AEM connectivity
-u username-i instance URL-p password
Rule 8 — Avoid including external custom dependencies
This can be harmful,hard in use and notportable!
Rule 9 — Keep It Simple Stupid
KiSS
Keep toolkit filesas small as possibleup to 400 LOC
Manage complexity!Simplify logic!
Rule 9 — Keep It Simple Stupid
KiSS
Keep toolkit filesas small as possibleup to 400 LOC
Manage complexity!Simplify logic!
Thank you!
Q&A
Resources
https://github.com/Cognifide/CQ-Unix-Toolkit
Presentation symbols provided by Tango Desktop Project
blog entry on Cognifide website