SITPLUS Manual
==============
:toc:
:numbered:
:lang: en
:encoding: utf-8
:website: http://sitplus.appctarragona.org

image::images/sitplus_logo_120x120.png[]

image::images/sitpluslogo_med.png[SITPLUS Logo]


Introduction
------------
SITPLUS is a free software framework whose main goal is to provide 
ludic-therapeutic activities for people with multiple disabilities. It offers 
new forms of interaction based on computer vision, voice and other peripherals 
to produce a result in the form of image and sound. Inspired by the cause and 
effect applications, SITPLUS provides a tool for continuous and remote
interaction, attainable to the majority of people with multiple disabilities.

How to install
--------------

Requirements
~~~~~~~~~~~~

- Dual core processor running at 1.5Ghz or higher.
- Sound card and speakers.
- Microphone (optional, for voice based interaction).
- Web camera (high quality webcam recommended).
- Wii Remote + Bluetooth adapter (optional, for Wii Remote based interaction).
- (Optional but recommended.) Good quality sound system and secondary display
or projector.
- Pure Data (pd) extendend version (pd-extended)
- Supported operating systems:

	* Microsoft Windows XP SP3/Vista/7
	* GNU/Linux

Installation process
~~~~~~~~~~~~~~~~~~~~

You can download the latest SITPLUS version from
http://sitplus.appctarragona.org[SITPLUS website]. A Windows installation
package is provided. For GNU/Linux you might need to fetch the sources and
compile them.

1. Before installing SITPLUS first uninstall any previous version.

2. Install Pure Data (pd).

	* *Windows.* Download and install *with the default options*
http://downloads.sourceforge.net/pure-data/Pd-0.41.4-extended.exe[Pd-extended]

	* *GNU/Linux.* Major distributions provide pre-compiled pd packages. 
Unfortunately, most pd packages do not provide all required components
(externals in pd jargon) which are required by SITPLUS and thus voice based
activities are currently unsupported.

3. Install SITPLUS.

Running SITPLUS
---------------

Before running SITPLUS make sure that the required devices such as sound input
(i.e. a microphone) and output, web camera, Wii Remote, etc. are already 
installed and work properly. Set also the system sound volume (both for 
recording and playback) accordingly. 

Once started check the configuration options under the _''Configuration''_
menu. To use a Wii Remote you must pair it with the computer, check
<<wii-remote-configuration,Wii Remote Configuration>> for further details. 
Check also how to configure Pure Data. <<pd-config, Pure Data Configuration>>.

Currently, activities can be started either by choosing _''Start...''_ or
_''Load file...''_ under the _''Activity''_ menu. The former option allows to
select which input channels to use (camera, voice and/or Wii Remote) before
starting one of the built-in the activities. The later option allows to load a
.sp file (some .sp files are provided under sp/ folder).

[[built-in-activities]]
Built-in activities
~~~~~~~~~~~~~~~~~~~

Although SITPLUS is conceived as a framework which allows for several ways 
of customization, it currently provides some built-in activities which combine
different input methods (camera tracker, voice input and Wii Remote based 
motion sensing) with real-time MIDI sound generation and a graphical collage.

.Activity screenshot
image::images/activity.png[Activity screenshot]

The screenshot depicts an activity which combines the use of the three input 
channels is shown. Using this activity the user can play a MIDI musical score
though the motion picked up using the camera and/or a wiimote. 
Motion is also use to make the graphical collage evolve. At the same time,
the user's voice is also picked up, processed using the chosen effects and 
played it back to the loudspeakers. Moreover, user's voice also makes the
collage evolve.

The window is organized in three main areas:

- The left area holds input channel options organized in a notebook.
- The central area shows the MIDI sequencer.
- The right area holds the graphical output controls.

Configuration
-------------

[[midi-configuration]]
MIDI Output Configuration
~~~~~~~~~~~~~~~~~~~~~~~~~

To set up the MIDI output to use, select ''Configuration'' and then ''MIDI 
Out...''. The configuration dialogue allows to choose and test a MIDI device.

If you do not have an external MIDI module attached to your computer or a 
similar device (which is the most common scenario), you need a software 
synthesizer in order to play MIDI sounds.

*Windows* ships a software synthesizer called "Microsoft GS Wavetable Synth".
TIP: to reduce the latency of this synthesizer take a look
http://ocremix.org/forums/showthread.php?t=23223[here].

On *GNU/Linux* most distributions provide binary packages for an high quality 
MIDI synthesizer called TiMidity++.   

To install TiMidity++ on Debian/Ubuntu run: 

	sudo apt-get install timidity freepats

TODO: http://ubuntuforums.org/showthread.php?t=789578
http://www.pulseaudio.org/wiki/PerfectSetup

[[wii-remote-configuration]]
Wii Remote Configuration
~~~~~~~~~~~~~~~~~~~~~~~~

To be able to connect a Wii Remote device (aka wiimote) - currently the
standard Wii Remote (also with Motion Plus) and the Balance Board are 
supported - you need a bluetooth adapter. The configuration of such device is 
beyond the scope of this manual.

On Windows you first need to pair you wiimote with the computer, see 
<<pairing-wiimote,Pairing the wiimote>>.

To check if the wiimote is properly detected access Configuration -> Wii
Remotes and press the ``Reconnect'' button (on GNU/Linux you might need to 
simultaneously press buttons ``1'' and ``2'' on the wiimote before
reconnecting). Once connected the wiimote will rumble and the dialogue will
be updated.

.Wii remotes configuration screenshot
image::images/wiimotesconfig.png[Wii remote screenshot]


[[pairing-wiimote]]
Pairing the wiimote (Windows only)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The basic steps are:

1. Open the Bluetooth Device configuration. Depending on your Windows version 
you will find the Bluetooth icon in the Windows control panel or in the task 
bar. A list containing the currently paired devices will appear.

2. Remove any previously paired device.

3. Select ``Add new device'' to start searching for available Bluetooth 
devices nearby.

4. Simultaneously press buttons ``1'' and ``2'' on the wiimote (LEDs will 
start flashing, the number of flashing LEDs also tells you the state of
the batteries).

5. Select the newly detected device and pair it without using codes.

If the process succeeded, LEDs will keep flashing. Note that after a system 
reboot you need to repeat this process again.

Sometimes (if the wiimote was not properly shut down) you might need to reset
it by removing the batteries and pressing ``1'' and ``2'' before putting them 
into again.

[[pd-config]]
Pure Data Configuration
~~~~~~~~~~~~~~~~~~~~~~~

1. Open the audio configuration dialogue (Configuration -> Pure Data).

2. Once opened you should hear a continuous sound. If no sound is present
make sure you have previously closed other applications dealing with sound
(e.g. media players, web browsers, etc.) and that system sound settings are
properly set. *NOTE:* on Windows Vista/7 it is advised to *not to use* ASIO
drivers on Pure Data, otherwise MIDI sound won't work. For further information 
about Pure Data check http://crca.ucsd.edu/~msp/Pd_documentation/[Pd
documentation].

3. Set ``Delay'' to the minimum possible value before you get sound defects.
You can also check if the microphone works properly by selecting 
``Microphone''.

.PD configuration dialogue screenshot
image::images/pdconfig.png[PD configuration dialogue]


Customization
-------------

SITPLUS allows different ways of customization ranging from creating new
musical scores or graphical collages to developing your own components.

What can be customized?

- *Musical scores.* The `scores/` directory contains musical scores that the 
currently provided activities use. To add new scores simply copy the file 
containing the new score to this directory. The score file format is quite 
simple. It consist on a text file where each line contains up to 4 notes 
(each note is represented by its MIDI number). Optionally the first line might
contain the word `percussion` to select the percussion kit.

- *Graphical collages.* The `graphics/mod_collage_xml/` directory contains a 
collection of collage definition files. These are XML files which describe the 
behaviour and contents of the graphical collages. See 
link:mod_collage_xml_help.html[Structure of XML file for mod_collage] for
a detailed description.  Under `graphics/mod_collage_xml/` 
there are two folders:

	* `activities/`: it holds the activities which will be draw in the  
foreground.
	* `backgrounds/`: contains the background definitions.

- *Activities.* The activity definitions can be found under the `sp/` 
directory. It contains scripts whose format is specified in link:activities.html[SITPLUS script syntax].


Information for developers
--------------------------

SITPLUS is built around a core C++ library (spcore) which provides basic 
services such as module loading, component instantiation, basic types and 
internalization among others, and base classes to develop new components.

The basic building block is the component (IComponent interface). A component 
is a processing unit which provides input and/or output pins and an optional
UI panel. Components can be arranged in a composite and its pins can be
connected to build data-flow oriented applications.

Along with with the core, two helper libraries are also provided:

- sphost: provides some helper functions and classes to develop final 
applications. It provides, for instance, a script parser to ease the creation
of component composites.
- widgets_base: provides UI base classes

Finally, several modules are also provided:

- mod_camera: components and types to deal with camera capture.
- mod_io: components for file I/O.
- mod_midi: components and types for MIDI output.
- mod_sdl: components and types to draw on SDL surfaces.
- mod_vision: components for computer vision.
- mod_widgets: UI components such as buttons, combo boxes, etc.
- mod_wiimotes: components and types to use Wii remotes

One of the goals of SITPLUS is to become GUI toolkit agnostic. Currently 
it uses wxWidgets and most modules provide UI panels based on this toolkit,
but the core itself has not an strong dependency (component's method GetGUI
expects wxWindows types though this is easily changeable by just defining a
neutral window type, the core itself provides methods such as InitGUISupport,
RunMessageLoop and CleanupGUISupport which are based on wx).

(This section is a draft and needs to be extended).

Project origin
--------------

SITPLUS was born as a research project in the Cerebral Palsy Centre 
http://www.appctarragona.org/[APPC] of Tarragona (Spain) in collaboration 
with http://www.crea-si.com[CREA Software Systems] and others. 
SITPLUS is the result of more than three years of research, 
development and direct intervention with people with cerebral palsy in 
the APPC. Inspired by the cause and effect applications, SITPLUS 
provides a tool for continuous and remote interaction, attainable to the
 majority of people with multiple disabilities. It was conceived in the 
APPC in late 2007 to try to provide a tool to the group of people whose 
combination of motor, cognitive and even sensory limitations prevented 
them from accessing other commonly used tools. Experimental sessions, 
conducted in the laboratory that the APPC has devoted to this project, 
show that this tool not only promotes participation, engagement and play
for many people with moderate to severe impairments, but is also very 
motivating for users with mild impairments.

image::images/sitpluslab.jpg[SITPLUS laboratory in the APPC]

Although SITPLUS focuses on people with cerebral palsy, we believe 
that it could benefit other people with cognitive disabilities. 
Moreover, it can also be useful as a platform for game or interactive 
music and visual arts research. Indeed, SITPLUS is conceived as an 
application framework in which several activities using resources run. 
We call resources things like input sound analysis, computer vision 
based motion tracking, drawing and sound generation algorithms, and so 
on. We call activities these pieces of software that employ resources to
do something useful (e.g. let someone interact in some way). The idea 
behind this model is to allow reusing those components to provide rich 
interactive scenarios whilst keeping simple its further development and 
usage.
