bci for beginners
TRANSCRIPT
IBM Brand Template
Iain Lewis 30 Sept 2013
BCI for Beginners
Important Disclaimers
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILST EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. ALL PERFORMANCE DATA INCLUDED IN THIS PRESENTATION HAVE BEEN GATHERED IN A CONTROLLED ENVIRONMENT. YOUR OWN TEST RESULTS MAY VARY BASED ON HARDWARE, SOFTWARE OR INFRASTRUCTURE DIFFERENCES.ALL DATA INCLUDED IN THIS PRESENTATION ARE MEANT TO BE USED ONLY AS A GUIDE.IN ADDITION, THE INFORMATION CONTAINED IN THIS PRESENTATION IS BASED ON IBMS CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM, WITHOUT NOTICE. IBM AND ITS AFFILIATED COMPANIES SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: - CREATING ANY WARRANT OR REPRESENTATION FROM IBM, ITS AFFILIATED COMPANIES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS
About me
Iain Lewis
QA Engineer, IBM Java Technology Center, IBM Hursley, U.K.
13 years experience developing and deploying Java SDKs
Contact [email protected]
Visit the IBM booth #5112 and meet other IBM developers at JavaOne 2013
Agenda
Who am I, and why am I here
What is BCI?
BCI and the JVM
Practical BCI
Who am I?
Test IBM's Java SDKI break Java
Focus on system testing in QAMulti-threaded load testing
Third-party applications
System testing the Java SDK means writing Java
Can the VM handle any application or Java code?
Recently wrote some BCI based testing
Why are you here?
Have heard of BCI
Would like to know more aboutWhat BCI is
What BCI can do
Would like practical help onHow to get started
Where to get help
What tools are available
As the title of the session suggests, the idea was to get together people who are interested in BCI, but don't yet know where they are going with it.
I'm not a world expert!Can help you avoid some of the pitfalls(new lines in jar manifest files)
Suggestions for tools to help you
What is BCI?
Bytecode Instrumentation is modifying a Java class without the source
Can be done offline, at load time, or at any point afterwards, and multiple times
Application and most SDK classes can be changed
Class File
BCI
Modified ClassJVM
Disk/Network
Runtime
Bytecode technically refers to the opcodes in a class, that is the method bodies. However, the whole class file can be modified
BCI can be added/removed at will while an application is running
Why BCI?
Support for BCI at runtime is built into the Java SDK
Java bytecodes are well documented and easy to work with
BCI is supported by many tools
BCI is widely used
Not a hack,Been available since java 5
What can you do with BCI?
Offline/at load timeReplace the entire class with a different one!
Add/Remove fields and methods
At any timeModify the logic of existing methods
Insert new logic into existing methods
Allows you to doProfiling/monitoring
Code coverage
Code generation
Runtime logging
At load time can do almost anything:Restrictions: You can't go mad! If you removed a method from a class which implements an interface, the class would no longer verify, and would fail to load
Some things can't be done whilst running
maybe allow you to replace a newer version if for some reason it was difficult to get a new version/recompile
What can you do with BCI? (cont)
public void service(ServletRequest req, ServletResponse res) throws ServletException, java.io.IOException{// insert new logic here
// existing method logic}
Examples: Monitoring, logging/tracing, profiling
Servlets running in an application server (tomcat, websphere, glassfish)
Running the server, no idea what the application doesCan inject instrumentation at the top of this method to recordURL, time taken, where the request has come from..
Why BCI?
Doesn't have to be compiled in to a class
Doesn't need access to the source
Can be turned on and off
Can be applied to an arbitrary application
When turned off, has no performance impact
There are other mechanisms for doing some of the things mentioned previously
Why do them using BCI?
How do you do it? JVM interface
BCI is supported at the JVM level
Supported via 'agents' and the Instrumentation interface
java -javaagent:/path/to/agent.jar[=options] MyClass
Native agents also possible via JVMTI
My main focus is dynamic BCI, where the classes are modified either as the JVM loads the classes, or whilst an application is running.
Agents can be written in c, or java.Java agents are easier to deal withJVMTI has a larger interface, allows your agent to do more than just BCI
How do you do it? JVM interface (cont.)
import java.lang.instrument.Instrumentation;public class MyJavaAgent { public static void premain(String agentArgument, Instrumentation instrumentation) {} }
Manifest-Version: 1.0Premain-Class: MyJavaAgentCan-Retransform-Classes: true
Remember the new line at the end of the manifest!
How do you do it? JVM interface (cont.)
Allows your agent to:Register as a ClassFileTransformer
Transformers get a call back when a class is loaded, in order to transform it
Can also trigger retransformations via retransformClasses()
Documentation of Instrumentation :http://docs.oracle.com/javase/7/docs/api/index.html?java/lang/instrument/Instrumentation.html
voidaddTransformer(ClassFileTransformer transformer)
A class file transformer gets a call back whenver a new class is loadedRetransform is how you trigger modifcation of an already running class
How do you do it? JVM Interface (cont)
A class is an array of bytes
A transformer returns a modified copy of the byte array
Class byte arrayfrom diskTransformer 1Copied/Modifiedbyte arrayTransformer 2Finalbyte arraypublic static byte[] transform(ClassLoader loader, String className, Class classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) {}
All transformations boil down to:Take an array of bytes, copy and modify the array, give it back to the JVM
How do you do it? Modifying Bytes (the hard way)
Read the Java Virtual Machine Specificationhttp://docs.oracle.com/javase/specs/jvms/se7/html/
Parse the class byte arrayhttp://docs.oracle.com/javase/7/docs/api/java/io/DataInputStream.html
Return a modified version
Offline- all of the above would be done by reading in a class file into a byte array
How do you do it? Modifying Bytes (the hard way)
DataInputStream is = new DataInputStream(new ByteArrayInputStream(classFileBuffer))
int magic = is.readInt();System.out.println("Magic number is " + magic); // 0xCAFEBABE
int minor = is.readShort();System.out.println("minor number is " + minor); // 0 for Java 7
int major = is.readShort();System.out.println("major number is " + major); // 51 for Java 7
SummaryJava SDK provides a mechanism for transforming classesGives you a a byte arrayNot very much other helpNow, need some help to actually twiddle the bytes
How do you do it? Modifying Bytes (the easier way)
Use a frameworke.g. ASM http://asm.ow2.org
Use javac/javap
Use a bytecode plugin for your IDEBytecode outline plugin for Eclipse http://andrei.gmxhome.de/bytecode/index.html
Write as much of your code in Java as possible
The following slides focus on making it easy to add new logic to an existing classOther frameworks are availableJavassist, bcel
n.b A basic understanding of how a JVM works is still required
Using a Framework (ASM)
Provides an abstraction layer to hide the details of a class
Easier to learn (provides symbolic constants, helper APIs)
Widely used
Under active development
Is an additional dependency
Doesn't tell you what bytecodes to create!
Easier to learn, but non-trivial java api
Does provide helper methods to do make common tasks easier(e.g. local variable sorter)
Now give brief overview, No details emphasis, this is just java, and there are only 3 classes and three methods
A transformer in 3 classes and 3 methods class 1
/** Implementation of transform() in ClassFileTransformer */public byte[]transform(...) throws IllegalClassFormatException{ClassWriter writer = new ClassWriter(ClassWriter.COMPUTE_FRAMES);ClassAdapter1 adapter = new ClassAdapter1(writer);ClassReader cr = new ClassReader(classfileBuffer);cr.accept(adapter, ClassReader.SKIP_FRAMES);return writer.toByteArray();}
This is class one, create a new instance of a method adapter
A transformer in 3 classes and 3 methods class 2
/** Override of visitMethod in ClassVisitor */public MethodVisitor visitMethod(...) {MethodVisitor mv = cv.visitMethod(...);return new MethodAdapter1(mv);}
In method adapter, override one method, create a MethodAdapter class
A transformer in 3 classes and 3 methods class 3
/** Override of visitCode in MethodVisitor */public void visitCode() {mv.visitCode();// Use ASM api calls to add profiler logic here}
This gets you to the point of:My class is parsed, I have a method body to create/modifyASM provides methods to add bytecodes butDoesn't tell you what bytecodes to add!Knowledge of the JVM architecture is still required
Next step, make creating new logic easier
Writing bytecodes with javac/javap
Writing logic in bytecode is possible, but not simple
Get your computer to do it for you!
Write your logic in Java
javac turns Java code into bytecode, packaged up in a class file
javap -v turns a class file into a human-readable format
Why javap? (and why not)Shipped with the Java SDK
Easy to use
Shows you the whole of the class, not just methods (helpful for learning)
Not easy to translate javap output into the Java code to produce it
Javap on HelloWorld
public static void main(String[] args) {System.out.println(Hello World);}
javap -v HelloWorld.class
Javap on HelloWorld
(...)public static void main(java.lang.String[]); flags: ACC_PUBLIC, ACC_STATIC Code: stack=2, locals=1, args_size=1 0: getstatic #16 // Field java/lang/System.out:Ljava/io/PrintStream; 3: ldc #22 // String Hello World 5: invokevirtual #24 // Method java/io/PrintStream.println:(Ljava/lang/String;)V 8: return LineNumberTable: line 7: 0 line 9: 8 LocalVariableTable: Start Length Slot Name Signature 0 9 0 args [Ljava/lang/String;(...)
This is just part of the output of javapAlso includes the whole of the constant poolEvery method in the classAll the meta data (stack maps, line number tabel, variable table)Not much help to go rebuild a similar method in your agentBut I do recommend it as a help to understanding how the JVM works
Writing bytecodes with the Bytecode Outline Eclipse Plugin
Plugin for Eclipse
Turns Java source code into bytecodes
Also shows you the Java code using ASM API calls to create the equivalent bytecode
(demo)
Demo
Very simple counting profiler
Prints out the number of times each method in your application was called
(demo)
Show the profiler logic in java show the bci logic in java Show byte code plugin to turn that into ASM (show view, other, byte code, use switch to change between asm and non-asm) Paste ASM code into boiler plate method transformer
Recap
Use a frameworke.g. ASM http://asm.ow2.org
Get your computer to write your bytecode for youUse javac and javap
Use the Bytecode Outline Plugin for Eclipse (or similar)
Write as much of your code in Java as possible
BCI is very powerful and used in many products
JVM knowledge is required but...
There are many tools available to make it easier
The following slides focus on the task of - adding new logic to an existing class
n.b A basic understanding of how a JVM works is still required
http://ibm.co/JavaOne2013
IBM booth #5112Wednesday 1pm - 3pm
References
Eclipse Bytecode Outline plugin (allows you to use asmifier in Eclipse)http://andrei.gmxhome.de/bytecode/index.html
ASMhttp://asm.ow2.org/index.html
java.lang.instrument package documentationhttp://docs.oracle.com/javase/7/docs/api/index.html?java/lang/instrument/package-summary.html
References
Inside the Java Virtual Machine Bill Vennershttp://www.artima.com/insidejvm/ed2/jvm.html
Java VM Spechttp://docs.oracle.com/javase/specs/jvms/se7/html/index.html
blue-logo
2013 IBM Corporation
IBM Confidential
23 September 2013
blue-logo 2013 IBM Corporation
37-degree-pos-tri-logo
blue-logo 2013 IBM Corporation
BCI for Beginners
23 September 2013
Click to edit the title text format
Click to edit the outline text formatSecond Outline LevelThird Outline Level
2012 IBM Corporation
blue-logo
2013 IBM Corporation
23 September 2013