Java Performance Monitoring and Benchmarking

Lengauer 1KV Block Start: 12.3.2021


This course teaches best practices in performance monitoring and benchmarking of Java applications. We will look at common traps when monitoring and benchmarking application as well as how to draw reasonable and sound conclusions based on the results. There will only be two blocks, the rest of the year the students have to work on two exercises. There will not be any exam.

Useful (but not required) lectures: Compiler Construction (Übersetzerbau), System Software, Systems Programming (Systemnahe Programmierung) (or similar courses)

Dates

<
Date Time Room
Fr, 12.03.2021 08:30 - 11:45 online
Fr, 12.03.2021 13:45 - 15:15 online
Fr, 23.04.2021 08:30 - 11:45 online
Fr, 23.04.2021 13:45 - 15:15 online

Contents

  1. Java VM Internals 101
    - Just-in-time Compilation, Optimizations, Garbage Collection, Safepoints, Locking
    - Useful profilings flags
    - Examining bytecode and machine code
    - Java Performance Pitfalls and Misbeliefs

  2. Performance Monitoring
    - Building custom java monitoring agents - Instrumentation, JMX
    - Building custom native monitoring agents - JVMTI, Sampling

  3. Benchmarking
    - What to measure? (Metrics)
    - How to measure? (Preparing the application under diagnosis and its environment)
    - How to analyze the data?
    - How to draw sound conclusions?
    - Hidden Factors

Handouts

The slide copies as well as other material can be downloaded from the Kusss page of this course.

Exercises

There are two exercises to be done by a team of at most 2 students. (detailed assignments will be provided via KUSSS)

1. Exercise:
Choose one of the following assignments:
1) Implement a Java profiling agent which detects hot methods.
2) Implement a Java profiling agent which prints the number of objects per class currently in the heap.
3) Implement a Java profiling agent which detects deadlocks.
4) Implement a Java profiling agent which detects the largest live sets.
x) Anyhing comparable to assignment 1-4.

2. Exercise:
Choose one of the following assignments:
1) Measure and compare performance figures using benchmark suites.
2) Measure and compare the performance of at least two different algorithms for the same problem.
3) Measure and compare the impact of different experimental setups.
4) Measure whether computers have moody days. :-)
x) Anything comparable to assignment 1-4.

A more detailed description of the individual assignments will be available soon.

Exam

There will not be an exam!

Literature

  • D. Lilja, Measuring Computer Performance: A Practicioner's Guide, Cambridge University Press 2005
  • R. Jain, The Art of Computer Systems Performance Analysis, Wiley 1991
  • 9 Common Fallacies in Java Performance
  • Java 8 API docs
  • Java Native Interface (JNI) API docs
  • Java Virtual Machine Tool Interface (JVMTI) API docs
  • Java Virtual Machine Management Extensions (JMX)
  • javassist - A bytecode instrumentation library
  • Apache BCEL - A bytecode instrumentation library
  • ASM - Yet another bytecode instrumentation library
  • Java Bytecodes
  • J. Shirazi, Java Performance Tuning - Efficient and Effective Tuning Strategies, O'Reilly 2000
  • G. McCluskey, Thirty Ways to Improve the Performance of Your Java Programs, 1999