On 09/14/2013 10:07 AM, Jonathan Protzenko wrote:
> Hi Jon,
> 
> It's a known issue (see « known issues » at 
> http://protz.github.io/ocaml-installer/); mostly, the installer 
> software that I'm using gets reported as a virus by every single 
> antivirus out there. I haven't found a solution to the problem, except 
> for reporting false positives to the antivirus editors, which is a 
> tedious an manual task.
> 
> If you have any idea as to how this can be improved, I'd be really 
> happy to hear it, since this problem has been bugging everyone.

TL,DR: try stripping your executable, apparently the debug info tripped some heuristics; use online virus scanner to check scan results from multiple vendors.
i.e. try this for ocamltopwin:
-       $(MKEXE) -o ocamlwin.exe $(OBJS) $(LIBS) $(EXTRALIBS) -subsystem windows
+       $(MKEXE) -o ocamlwin.exe $(OBJS) $(LIBS) $(EXTRALIBS) -subsystem windows -link -s

I used to work on the open source Clam Antivirus, so here are some tips:

1. Scan your executable / installer with ClamAV

In this case ClamAV doesn't detect anything, so skip this step.
If it would detect something you could use it to narrow down which file causes the detection exactly.

2. Scan your executables / installers with multiple AVs online:
https://www.virustotal.com/en/
http://virusscan.jotti.org/

Note: obey the file size limits.
If it says the file is already known, tell it to reanalyze to get up-to-date results.
You can use this to:
 * check if any vendor currently detects your executable as malicious (preferably before public release)
 * check when/if they fix the false positive
 * narrow down which file is triggering the false positive

You can also use this to narrow down which file is malicious, for example: you can zip directories
to narrow down which directory contains the malicious executable, and then try uploading individual executables.

They also have a public API, but with a rate limit of 4 request / minute:
https://www.virustotal.com/en/documentation/public-api/

Note: These online scanners automatically share the samples with AV vendors, so it might happen that this will cause yet more detections later on.
Other vendors are not supposed to blindly follow another vendors' scan results, but you never know ...

3. Once you found your suspicious executable investigate it with 'clamscan --debug'

It is an open-source tool (apt-get install clamav), and can help you understand how an AV would "see" your executable.
Ignore everything in the debug output until this line: LibClamAV debug: in cli_magic_scandesc

From step#2 I found that OCamlWin.exe causes the most detections, Detection ratio: 	17 / 46 

In this case ClamAV does not detect the file, so I can only guess why other AVs detect it but there are some interesting things in the output of 'clamscan --debug OCamlWin.exe':
LibClamAV debug: ishield: @3440a found file .file () - version  - size 1
LibClamAV debug: ishield: extracted to /tmp/clamav-d8556f929d4a33a9ea6cbecdb800b9cc

Now OCamlWin.exe is not an InstallShield file, so why did it try (and find) something installshield-like in it?
The file has what ClamAV calls overlays: i.e. extra data between the end of the executable and the end of the file.
It doesn't know what is there so it tries various heuristics.

Looking at the PE header table earlier in the debug this is the last section:
LibClamAV debug: Section 18
LibClamAV debug: Section name: /124
LibClamAV debug: Section data (from headers - in memory)
LibClamAV debug: VirtualSize: 0x8f0 0x1000
LibClamAV debug: VirtualAddress: 0x47000 0x47000
LibClamAV debug: SizeOfRawData: 0xa00 0xa00
LibClamAV debug: PointerToRawData: 0x33a00 0x33a00

end of executable = 0x33a00 + 0xa00 = 0x34400 = 214016
end of file = 285026

Now look with a hex editor in that area and you see things like .text,.data,.bss, etc. so it is not unreasonable to assume that
this might've confused some AV's heuristic which thought that you embedded some other executables there.
At a closer look it seems to be some kind of debug info (but why is not referenced from a PE section?), which is also hinted by the large number of sections in the executable.

So I tried this: i686-w64-mingw32-strip OCamlWin.exe -o y.exe

Uploaded to virustotal again and things have improved: Detection ratio: 	5 / 47

4. Try different compilers

I have tried to cross-compile ocamltopwin from Linux (using ocaml 4.00.1, thats the latest cross-compiler I have), and I got 0 detections...
FWIW on Linux I did this:
* apply linux.patch
* FLEXLINKFLAGS="-L/usr/i686-w64-mingw32/lib/" CONFIG=/path/to/ocaml/sources/Makefile.mingw make

So either something changes in ocaml 4.01.0 thats triggering some AV heuristics, or your windows build environment is sufficiently different that
causes heuristics to trigger.
FWIW I have:
$ i686-w64-mingw32-ocamlc -version
4.00.1
$ i686-w64-mingw32-gcc --version
i686-w64-mingw32-gcc (GCC) 4.6.3

5. Some other things you can do

If you can afford a code signing certificate then sign your installer / problematic executable.
Note that signing just the installer will not prevent false positives once the application is extracted and installed.

Best regards,
--Edwin