An illegal opcode, also called an undocumented instruction, is an instruction to a CPU that is not mentioned in any official documentation released by the CPU's designer or manufacturer, which nevertheless has an effect. Illegal opcodes were common on older CPUs designed during the 1970s, such as the MOS Technology 6502, Intel 8086, and the Zilog Z80. On these older processors, many exist as a side effect of the wiring of transistors in the CPU, and usually combine functions of the CPU that were not intended to be combined. On old and modern processors, there are also instructions intentionally included in the processor by the manufacturer, but that are not documented in any official specification.
While most accidental illegal instructions have useless or even highly undesirable effects (such as crashing the computer), a few might by accident do something that can be useful in certain situations. Such instructions were sometimes exploited in computer games of the 1970s and 1980s to speed up certain time-critical sections. Another common use of them was in the ongoing battle between copy protection implementations and cracking. Here, they were a form of security through obscurity, and their secrecy usually did not last very long.
A danger associated with the use of illegal instructions was that, given the fact that the manufacturer does not guarantee their existence and function, they might disappear or behave differently with any change of the CPU internals or any new revision of the CPU, rendering programs that use them incompatible with the newer revisions. For example, a number of older Apple II games do not work correctly on the newer Apple IIc, because the latter uses a newer CPU revision – 65C02 – that does away with illegal opcodes.
More recent CPUs, such as the 80186, 80286, 68000 and its descendants or the PowerPC processor, do not have illegal opcodes that are widely known/used. Their manufacturers guarantee that the CPU will behave in a well-defined way when it finds an unknown opcode in the instruction stream: usually, this means triggering a certain exception or fault condition. The operating system's exception or fault handler will then usually terminate the application that caused the fault. Another, less common way of handling illegal instructions is by defining them to do nothing except taking up time and space (equivalent to the CPU's official NOP instruction); this method is used by the TMS9900 and 65C02 processor, among others.
In spite of this manufacturer guarantee against such instructions, research using techniques such as fuzzing has uncovered a vast number of undocumented instructions in modern x86 processors. Some of these instructions are shared across processor manufacturers, indicating that Intel and AMD are both aware of the instruction and its purpose, despite it not appearing in any official specification. Other instructions are specific to manufacturers or specific product lines. The purpose of the majority of x86 undocumented instructions is unknown.
Today, the details of these instructions are mainly of interest for exact emulation of older systems.
- Domas, Christopher. "Breaking the x86 Instruction Set". Retrieved 3 January 2018.
- Illegal opcodes on the 6502: http://www.ffd2.com/fridge/docs/6502-NMOS.extra.opcodes (see also How MOS 6502 Illegal Opcodes really work)
- Illegal opcodes on the Z80: http://www.myquest.nl/z80undocumented/