5

I have an executable that runs in the operating system, but when when I open it in $tool I get an error. What is going on?

asheeshr
  • 2,465
  • 8
  • 28
  • 41
Vehm Stark
  • 325
  • 1
  • 5
  • 6
    An example of an error (e.g. a screenshot) would make this more realistic. – Igor Skochinsky Apr 18 '13 at 18:29
  • Dunno why you were voted down. It's a worthy topic. Perhaps the question wasn't clear. – dyasta Apr 19 '13 at 08:31
  • 1
    @90h: the question is relatively clear, but Igor's point of criticism hasn't been addressed at all, but it would make sense to address it. – 0xC0000022L Apr 19 '13 at 14:38
  • Igor knows just as well as I do that many tools used to statically analyze files have bugs in their parsers. I wrote this question to share information for the benefit of others, but I don't time to find a broken tool and run it just to add a screenshot to the question. If Igor thinks it is necessary, I'd be happy to take an image furnished by him and add it to the question. – Vehm Stark Apr 20 '13 at 21:21
  • 1
    @VehmStark Let the community decide. I think it's fine as is. It is a general question with a general answer, and doesn't require specifics to have utility. – dyasta Apr 21 '13 at 11:22
  • 1
    A related question at RE.SE, including a couple *specific examples*, is here: How to Prevent Use of Resource Editors – dyasta Apr 21 '13 at 11:43
  • You question is too vague if you explain more about the type of tool I think that would help alot. – cb88 Jun 06 '13 at 14:43

2 Answers2

9

This is caused by a differences in the operating system's loader and the file format parsing code in the tool you are using. Malicious program authors often exploit differences between an executable file format's specification and how the file format is actually used by the loader in practice. If there are differences between the file format specification and the operating system loader, there exist executables which will run but are not legal according to the specification! For example, up until Windows Vista a PE executable can be missing several header fields and the Window's loader will still load it. It's likely your tool was written by looking at the file format specification and not at the implementation of the loader and thus cannot necessarily handle all executables which are successfully loaded by the operating system.

In reality an executable file format is specified in three categories of places:

  1. the formal file format specification
  2. the implementation of the operating system's loader
  3. the implementation of third party tools

These three things often differ in very subtle ways and malicious program authors take advantage of this fact. Just remember that the ultimate arbiter for whether something is a valid executable or not is the loader. If the executable runs but your tools are failing, your tools do not take into account the real-world behavior of the operating system loader.

The best work documenting this phenomena that I am aware of is by Ange Albertini for the PE executable file format.

Vehm Stark
  • 325
  • 1
  • 5
  • 2
    After Windows Vista, it's not possible anymore to cut the OptionalHeader at Subsystem: the comment in pefile is only valid until Windows XP. – Ange Apr 18 '13 at 20:20
1

Try using Rohitab's API Monitor to use the loader to open the program and watch the failed calls. Might point you in the right direction.

Mick
  • 7,562
  • 3
  • 26
  • 40
  • 1
    actually this wouldn't help, because the third-party programs handling PE files will not use the default library functions provided by the operating system, e.g. as part of ntdll.dll. – 0xC0000022L Apr 22 '13 at 12:50