If you don t control the environment, then I m sorry - you re really stuck. Yes, you could look for trivial JVMTI agents via some sort of cmdline sniffing, but that s the least of your worries. Think about java/lang/Classloader.defineClass() being compromised directly. That s easy to do if you own the box - just replace the .class file in rt.jar. In fact, until JVMTI came around, that was a typical way that profilers and monitoring tools instrumented Java code.

Going back to JVMTI - the "Late attach" feature also allows for JVMTI agents to be loaded on the fly. That might not have happened when you scanned the first time around.

Bottom line - if someone can change the bytes of the JRE on disk, they can do anything they want. Is it ethical, no? Can they get caught? Possibly, but you ll never win the war.

It looks like I can go with a combination of checks inside some custom JNI native code.

1.) cmd line sniffing to search for agents. 2.) Ensure that the cmd-line parameter -XX:+DisableAttachMechanism exists. (this will prevent people from attaching to my running VM)

I remember I once made almost a silent Java Agent. I guess you better look for port scanners or something around that.

Java 2 security, signing of jars etc, gives some level of control over what gets loaded into your application.

However in the end if a malicious person has access to a machine such that they can write to disk then in all probability they have plenty of capacity to do harm without resorting to clever Java hacks.

Turn this round, in any language what can you do to detect Trojans?

Careful access control to the machines you care about is non-trivial but essential if you are serious about such issues. Security specialists may seem paranoid, but that often means that they really understand the risks.

If you can t control the platform, you can t control the software upon it.

Even if you could shut down all the avenues of inspection you ve listed, Java is open source. They could just take the source code and recompile it with the necessary changes built-in.

Also, try to remember that while it is your code, it s their machine. They have a right to inspect your code to verify that running it on their machine does what they expect it to do, and doesn t perform "extra" actions which they might find undesirable. Less trustworthy companies in the past have scanned for non-relevant files, copied sensitive information back to their home servers, etc.

I would look at the command line and see, if there are any "-agent" parameters. All profilers, debuggers and other code modificators use this for introspection. You could also check for unusual jars on the bootclasspath, since those might also provide a threat (but be aware that you then also must deliver a custom JVM, since some software like Quicktime adds itself to the bootclasspath of ALL java apps running... (I couldn t belive my eyes when I saw that...))

Basically this is a loosing battle.

Have a look at how visualvm in the Sun JDK works and how it can attach to a running process and redefine whatever it pleases. It is extremely hard to detect that in a portable way, and unless you can do so, you might as well give up on this approach.

The question is, what is it you want to avoid?