Message-ID: <3D835866.3020705@csi.com>
Date: Sat, 14 Sep 2002 11:40:22 -0400
From: John Colagioia <JColagioia@csi.com>
User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.0.1) Gecko/20020823 Netscape/7.0
X-Accept-Language: en-us, en
MIME-Version: 1.0
Newsgroups: rec.arts.int-fiction
Subject: Re: [Inform] Tokenising strings
References: <Pine.GSO.4.31.0209101519390.22681-100000@barasson>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
NNTP-Posting-Host: ool-182f30fa.dyn.optonline.net
X-Original-NNTP-Posting-Host: ool-182f30fa.dyn.optonline.net
X-Trace: excalibur.gbmtech.net 1032017431 ool-182f30fa.dyn.optonline.net (14 Sep 2002 11:30:31 -0400)
Organization: ProNet USA Inc.
Lines: 15
X-Authenticated-User: jnc
Path: news.duke.edu!newsgate.duke.edu!nntp-out.monmouth.com!newspeer.monmouth.com!news.maxwell.syr.edu!nntp.abs.net!uunet!dca.uu.net!excalibur.gbmtech.net!not-for-mail
Xref: news.duke.edu rec.arts.int-fiction:108642

Christos Dimitrakakis wrote:
> I have a string, i.e:
> "The police station is just next to the bakery."
> I want to do two things:
> 1. Tokenize it.
> 2. Tag the words that are in the dictionary.
> I think there is a Tokenise__(source, destination) command in parserm.h
> but then, how do I find out which of the tokens are in the dictionary?

I'll double-check this, next time I'm at my laptop (where I do most
of my Inform work), but I believe all the low-level tokenizing and
parsing I've done in Inform, I've done with NextWord() and
NextWordStopped().  I may have had to dip into the...there's some
array that gets filled by the command line.

