
Native features proposal

Laurent Vogel, 8 december 2001


Introduction

This document describes a general technique to allow interaction between
software running on a m68k processor inside an emulator, and native
software.

In this document, the emulator is some software running on a native 
platform, to run software in a 68k platform. This document specifies
a general technique by which 68k software can interact with special
native software to achieve special effects not allowed directly on
the 68k platform.

To this end two interfaces are specified
- the 68k interface, describing special illegal opcodes by which the 68k
  code can call the native feature services.
- the implementation interface, describing how native features can be
  implemented in a partable way independently of the specific emulator.

This is illustrated in the following diagram.


                    native side                              68k side
   ______________________/\______________________           ____/\____
  /                                              \         /          \
            implementation                           68k 
              interface                           interface
   __________  (nf.h)   _________________________           __________
  |          |         |          | g |          |         |          |
  | feature  |<------->| native   | l | CPU and  |<------->| 68k code |
  | implemen-|         | feature  | u | memory   |         |__________|
  | tation   |         | services | e | core     |
  |__________|         |__________|___|__________|
                                  
                                  \______  ______/
                                         \/
                                    specific to  
                                    each particular  
                                    emulator


Scope and contents

The Native feature proposal comprises of mandatory specifications, optional
specifications, and optional implementations.

The mandatory specifications is the 68k interface, i.e. the behaviour 
that can be observed from 68k side regardless of how it is implemented in
the native side.

An optional set of specification is a basic set of native features,
that can be supplied or not supplied by the emulator.

Another optional specification is the implementation interface, i.e. the 
services provided by the emulator to implementations of particular native 
features. This specification is optional because it is perfectly valid to
implement native features without using this interface (for example for 
native features very tied to particular emulators).

Other optional specifications may describe Operating System-related
services on the 68k side to encapsulate the raw 68k interface. A suitable
additional specification for TOS is provided in annex E.

Optional implementations are
- a supplied implementation of the generic native feature services;
- supplied implementation of some particular native features;
- supplied glue to particular emulators
These are optional, because it is perfectly valid to provide the same
functionality using a different implementation.


Definitions

The 68k platform here is a virtual machine based on a motorola 680xx 
processor, where xx is one of 00, 08, 10, 20, 30, 40, 60, or any other 
processor with the same programming model and instruction set. 

Each native 'feature' is a group of 'functions', each taking arguments and 
returning a value.

here is a short list of terms that have a special meaning in this document:

  id - non-null feature identifier with bits 0-19 set to zero
  sub-id - 20-bit function identifier integer 
  feature - a groupe of functions
  function - a native function
  68k side - a virtual machine based on a supported processor
  supported processor - 68008, 680[012346]0, and compatibles
  special opcodes - 68k opcodes used by the native feature framework
  emulator - the container of both the 68k side and the native side
  basic set - the mandatory set of features


Limitations

Other 68k processors like the ColdFire, whose instruction set is not
compatible to the instruction set of the supported 68k processors, are
not taken into account by this document. In the event where some 68k
code were to be designed to support both the current specification 
(when running on a supported 68k processor) and unsupported processors,
that code should not use the special opcodes if it can be determined 
that the processor is unsupported, either by first checking the 
processor type, or by determining in an operating-system-dependant
way that native features are not available.

This document does not support emulators emulating a multi-processor 
68k machine (several 68k processors running in parallel).

The implementation interface, and the proposed implementation of native
feature services do not fully support multi-thread native environments
in which the 68k memory can be modified during the execution of one
native feature call.

All 68k memory accessed during the execution of a native function, 
either directly (the stack), or indirectly (following pointers) must
reside in physical memory before the native function is called. 
As it is an error to call a native function with non-accessible
memory, the implementation of native functions may ignore the 
possibility that bus errors occur when they fetch 68k memory. 
(so, they can let the 68k glue raise a bus error automatically, 
without testing beforehand that the memory is valid and without
having to take steps to ensure that no resource lossage occurs
due to such automatic bus errors being raised).

Some of these limitations might be solved by changing the implementation of
native feature services. 


Principle

Each native feature has a name uniquely identifying this feature.
68k code inquires whether the feature is available by asking for the
'ID' corresponding to the name:
 
  long id = nf_get_id(name);

If the returned ID is zero, then the feature isn't available. Else, 
functions provided by this feature may be called given this ID, a sub-ID
identifying the function among those provided by the feature, and 
the requested function arguments:

  long result_value = nf_call(id+sub_id, the, requested, arguments);

IDs have the following format:
- bits 20-31 hold a meaningless number internally managed by the emulator;
- bits 0-19 are set to zero, and serve as a place-holder to contain the
  sub-ID, a number that is given to the native implementation to allow
  efficient handling of multiple actual functions all regrouped under
  the same feature name.

See Annex G for a full example of how to call the basic set.


This document describes
- a small number of special opcodes (m68k illegal instructions) used 
  to manage native features;
- guidance for defining native features;
- a proposed basic set of native features that may be implemented by
  emulators.


Compliant emulator

Any emulator has the choice of providing the full interface described in
this document, or nothing.

If the emulator does not provide the interface, then all illegal
opcodes defined below must provoke the normal m68k behaviour 
(i.e. raise an illegal exception).

If the emulator provides one illegal opcode defined below, then it must
also provide the complete set of opcodes, with the behaviour defined in
this document. Such an emulator is called a compliant emulator.
In the rest of the document, 'emulator' means 'compliant emulator'.


C calling convention

Native functions do the job of C functions, using the C calling convention
described here:
- values are represented in memory as big endian (the byte with
  lowest address is the most significant byte);
- 'char' are 8bit values, either signed or unsigned (unspecified);
- 'short' are 16bit values;
- 'int', 'long' and pointers are 32 bit values;
- the return type of a function is either void (no return type),
  or a 32bit value; functions cannot return 8bit or 16bit values,
  and functions cannot return structs;
- parameters are passed on the stack; the last parameter is pushed
  first, then any parameters until the first parameter which is pushed
  last; when pushing parameters, if the size of the parameter is less 
  than 32 bit, the parameter is converted to a 32bit value by padding
  to the left with unspecified bytes;
- struct members are set in memory in growing addresses, with padding
  using the smallest number of unspecified bytes needed so that any member 
  not of type char, signed char, unsigned char, or an array of these 
  three types, has an even address.

String encoding is not specified in this document. Generally speaking
native functions should document this. When a string is to be displayed
to the user (as opposed as written in a file) it can be assumed that 
chars in the range 0x20 to 0x7e are ASCII (ISO-646).

Native functions should not alter any registers other than d0, unless 
properly justified and documented.


Naming

Feature names beginning with 'NF_' or 'nf_' are reserved.


Special opcodes

Special opcodes are taken in the range 0x73nn (illegal moveq instructions).
The following opcodes are defined:

  mnemonic nf_getid (in ARAnyM debugger it's NATFEAT_ID)
  value 0x7300
  prototype  unsigned long nf_get_id(const char * feature_name);
  ignore the first long word on the stack; let feature_name be the
  second long word on the stack; if the null-ended string pointed to
  by feature_name is the name of an implemented native feature, then 
  return its ID in register D0, else return zero.

  mnemonic nf_call (in ARAnyM debugger it's NATFEAT_CALL)
  value 0x7301
  prototype   long nf_call(unsigned long feature_id, ...);
  ignore the first long word on the stack; let feature_id be the second 
  long word on the stack; if feature_id is not a valid native function
  ID then the result is unspecified; else, call the function defined 
  by the ID (optionnally containing a non-null sub-ID), passing the 
  rest of the stack arguments (i.e. starting at address stack+8) as 
  parameters. 
  returns in register D0 the 32bit value returned by the fonction. 
  If the function is documented as not returning any meaningful value,
  (i.e. declared as void function(...)) then the value in D0 is 
  unspecified.

Note: both these special opcodes ignore the first long word on the stack. 
This is done so that these special opcodes can be encapsulated into small 
assembler routines that are declared with the same prototype as indicated
above for each special opcode. For instance, here is an actual declaration
and code for nf_get_id:

  extern long nf_get_id(const char * feature_name);

        .global _nf_get_id:
  _nf_get_id:
        dc.w 0x7300
        rts

and here is another implementation of the same:

  long nf_get_id_opcodes = 0x73004e75;
  #define nf_get_id ((long (*)(const char *))&nf_get_id_opcodes)


These opcodes are defined both for supervisor mode (Interrupt or
Master modes on 680x0) and normal user mode. Each NatFeat has a bool
flag "isSupervisorModeRequired" and if set to true then calling such
opcode from user mode will result in a priviledge exception.

The stack used for passing arguments will be the current stack
(SSP/ISP for Supervisor/Interrupt mode, MSP for Master mode).


Feature documentation

Feature documentation shall provide the following information:
- the feature name, a case-insensitive string of printable ASCII
  characters.
- the features that are required for this one to work
- the list of functions provided by this feature, and for each function:
  - its sub-id
  - a description of the way the feature works, including a function
    prototype and a description of return values and error cases

Optionally the feature text can include an implementation,
that is, source code running inside the emulator. In this case
it is recommended that it be given with a licence allowing it to
be included in most emulators, including closed-source ones
(licence like LGPL); also it is best if such sample implementation
is written in one of the proposed implementation interfaces.


Basic set features

"NF_NAME"
  sub-ID 0 - user allowed
  unsigned long getName(char *buffer, unsigned long size)
    fills 'buffer' with a null-terminated string representing the
    emulator name. At most 'size' bytes will be written in the
    buffer (which means that the string will be truncated if it
    contains more than size - 1 characters).
    the string should contain only printable ASCII characters
    (ASCII 32 to 126 inclusive).
    Returns the length of the string before truncation. 

  sub-ID 1 - user allowed
  unsigned long getFullName(char *buffer, unsigned long size)
    similar to getName, but the string is the emulator name plus 
    its version (and anything else that might be printed together 
    with the emulator name)
    Returns the length of the string before truncation.

"NF_VERSION"
  sub-ID 0 - user allowed
  long getVersion(void);
    returns the version of the native feature framework, hex encoded
    (upper word = major number, lower word = minor number). This document
    describes native features version 1.0 (hex encoding 0x00010000).

"NF_STDERR"
  sub-ID 0 - user allowed
  unsigned long output(const char *string);
    emit a null-terminated string of printable ASCII chars on a
    particular output stream, intended for use to display debug 
    messages. Returns the number of chars printed. Characters
    not among the set of ASCII chars 9 (horizontal tabulation '\t'), 
    10, 13, (both meaning '\n' newline), 32-126 will have 
    implementation defined representations.

"NF_SHUTDOWN"
  sub-id 0 - supervisor mode only
  void shutdown(void);
    immediately terminate the execution of the emulator, with no 
    user interaction.


Error cases

features should be developed in a way that allow them to be
called from multitasking host systems. To achieve this, there
should not be a single 'errno' feature collecting the status
of multiple distinct features, but rather each feature or
feature group should provide an independent way of reporting
errors.

In any case calling a feature with incorrect parameters should
not crash the emulator.


Memory

On emulators implementing MMU and where physical addresses
differ from logical addresses, the memory that will be
accessed by native features uses the logical addresses (that
is, exactly the same memory than that seen by the CPU).
This does not prevent special features to have special
behaviour.

If the feature was asked to access memory that does not exist, or whose
access is not allowed by the current MMU policy, or to write to ROM, 
then the feature should directly provoke a bus error. The exact address 
used in the bus error stack frame can be any address within the range
of offending addresses.


TODO: remove the need for reporting the offending address?
DISCUSSION:
  almost no emulator keeps track of the offending address,
  and even less 68k software cares for it. relaxing the need for
  reporting an offending address would simplify native feature 
  implementations

for instance, if a bzero native feature is asked to clear
from memory 0x2000 to 0x3fff, and only memory 0 to 0x2fff
is writeable, then
- there should be a bus error,
- the value of memory from 0x2000 to 0x2fff is 'unspecified'
- the address reported in the bus error can be any address
  between 0x3000 and 0x3fff

for features running on a 68000-only emulator, asking for
unaligned memory accesses will NOT provoke address errors.
However, such aligned memory accesses may be much slower.

DISCUSSION:
  We don't want to impose the emulator to implement unaligned 
  memory access; however we don't want to impose to the native 
  function implementation to check whether they are running
  on 68000-only emulators.


Calling 68k code from native side.

There is no provision in this document to allow native features
to call 68k subroutines. Therefore the control flow must reside
entirely on the 68k side. Feature designers should decompose the
native functions into small functions that can be interspersed, 
on the 68k side, by calls to the required subroutines. 


Appendix A - TODO list

- add a paragraph encouraging feature designers to think!

- add a paragraph encouraging native features using interrupts
  to be as independent from other features as possible, to
  avoid deadlock

- rework appendix B completely.



Appendix B - implementation interface

see file nf.h (to be rewritten following experience gained)


Appendix C - sample implementation

(to be rewritten following experience gained)


Appendix D - Contributors

Thanks to all who did contribute to this document, including:

Milan Jurik 
Petr Stehlik
Martin Döring
Johan Klockars
...

TODO: complete this list.

Appendix E - TOS glue : the __NF cookie

TOS-compatible operating systems may define a cookie named '__NF',
pointing to the following data:

  struct cookie__NF_data {
    long magic;  /* equals 0x20021021 */
    long (*getid)(const char *name);
    long (*nfcall)(long id, ...);
  };

this cookie serves two purposes:
a) report that native features are available
b) provide pointers to functions encapsulating the illegal opcodes.

It is intended to be used as follow:
- 68k code that do not intend to ever run on unsupported processors is
  not obliged to check for this cookie, and may instead use the 
  illegal opcodes directly; However it may check for the cookie, and
  if the cookie is not present it may still try the illegal opcodes.
  if the cookie is present it may use either the supplied function
  pointers or the illegal opcodes.
- 68k code that must be able to run on a unsupported processor must
  check for this cookie. In any case the illegal opcodes should NOT be
  used directly. If the cookie is not present, then no native features
  are available. If the cookie is present, then native features are 
  available, throught the use of the provided function pointers in the 
  cookie.


Appendix F - a small example

see file ../atari_example.c ???


Appendix G - modifications to this document


2001-12-08 
- 'native feature proposal' posted to aranym mailing list.
2001-12-09
- erratum about an interrupt manager (not integrated in this document)
2001-12
- discussions on the aranym mailing list; moving from 16 to 32bit ints,
  question about string charsets, replaced 'host side' by '68k side'.
2002-03-02
- rewriting of the document, put some things in attic.txt,
- rewrote the beginning of the document, added an overall diagram.
- put the implementation interface in a separate nf.h

2002-08-11 (joy)
- corrected the actual nf calls according to what is implemented in ARAnyM.
- NatFeats are usermode callable now.
- Value-only features removed.
- Basic set of features replaced with list of current basic NF implemented
  in ARAnyM.

2002-10-22
- enumerated supported processors, natfeat usage on unsupported processor
  need an OS-specific way (i.e. __NF cookie)
- removed direct call; opcodes better explained (ignored return address)
- told about sub-id, distinguished between 'features' and 'functions'
- reworded a little the basic set (TODO, check with actual aranym 
  implementation (joy?))

2002-10-28
- NF_STDERR added, NF_NAME corrected (returns the number of chars before
  truncating).

2002-10-29
- a note about valid m68k physical memory added

2003-06-03
- NF_SHUTDOWN added
