\start
Date: Sun, 30 Nov 2003 21:30:15 -0500
From: Tim Daly
To: Bertfried Fauser
Subject: Re: AXIOM from cvs does not compile

> TANGLE=/home/fauser/axiom/mnt/bin/bin/lib/notangle
                                ^^^^^^^
hummm... which means that SPADBIN must be bad:

> SPADBIN=/home/fauser/axiom/mnt/bin/bin

but this is a constructed variable ending in SYS:

> SYS=bin

Therefore the AXIOM variable must have been:

AXIOM=/home/fauser/axiom/mnt/bin

but should have been:

AXIOM=/home/fauser/axiom/mnt/linux

\start
Date: Mon, 1 Dec 2003 10:55:19 +0000
From: Mike Dewar
To: list
Subject: Re: Axiom Book (Jenks & Sutor)

Sorry for the delay in responding to people who asked for copies, but I
was out of the office at a meeting.  The winners are Juergen Weiss,
Michel Lavaud, Martin Rubey, Luca Giuzzi and Hans Peter Würmli, who
should receive copies shortly.  Sorry to the unlucky applicants, in the
unlikely event we find any more then I'll let you know.

Several people offered money for these, I should have made it clearer
that the books were free (we would just have thrown them away if a
good home wasn't available).

Kind regards, Mike.

On Fri, Nov 21, 2003 at 09:40:58AM +0000, Mike Dewar wrote:
> While clearing out a cupboard, one of my colleagues came across a
> handful of copies of Jenks & Sutor.  If anybody on this list would like
> one then please send me your snail mail address and we'll ship it to you
> wherever you are, even if its the Netherlands :-) [1]  First come, first
> served.
> 
> Cheers, Mike.
> 
> [1] this is a football reference. Holland trounced Scotland on
>     Wednesday :-(

\start
Date: Mon, 01 Dec 2003 22:53:16 +0100
From: David Mentre
To: Tim Daly
Subject: Re: Questions (Bugs?)
Cc: Stefan Dirnstorfer

Tim Daly writes:

> Please add it as a bug so it doesn't get lost.

Ok. I've added Stephan's third bug as #6903.
http://savannah.nongnu.org/bugs/index.php?func=detailitem&item_id=6903

\start
Date: Mon, 01 Dec 2003 22:32:40 +0100
From: David Mentre
To: Tim Daly
Subject: Re: About pamphlets and literate programming: how to identify identification levels?

Hello Tim,

Tim Daly writes:

> I'm currently working on a demonstration pamphlet based on the paper called
> Primes is in P. I'm hoping to finish it and present it at a workshop in
> Italy next week. It is an ideal example of working from the theory all
> the way down to the code.

Ok. I'm not sure now that I made my point very clearly but I'll wait
until you make available this paper and I'll ask you further questions
if needed at that time.

\start
Date: Mon, 1 Dec 2003 17:49:52 +0100 (CET)
From: Bertfried Fauser
To: Tim Daly
Subject: Re: AXIOM from cvs does not compile
Cc: Bertfried Fauser

On Sun, 30 Nov 2003, Tim Daly wrote:

Dear Tim,

sorry for my stupidity,

> Therefore the AXIOM variable must have been:
>
> AXIOM=/home/fauser/axiom/mnt/bin

your guess was quite right, even if the AXIOM variable was different, it
was not OK. I wonder why I was able to compile AXIOM the first time....

All in all, a few other PATH issues seemed to fail so that AXIOMsys was
not functional. Its fixed and --heureka- this solves my TeXmacs problem
too.

\start
Date: 01 Dec 2003 17:47:19 -0500
From: Camm Maguire
To: Tim Daly
Subject: Re: [Gcl-devel] [Re: Executable memory: further programs that fail]

Greetings!  Tim, the -Wa,--execstack option is trivial to implement
and is sure to work.  It would be nice to have a configure test to
determine when it is necessary, but all gcc I have available seem to
take the option.  Even though it is not the desirable long term option
as the poster states, a final solution must await a fix to emacs'
unexec, at which point we can move DBEGIN to an internal variable, and
test if holes are permitted in the heap.  So at this point I recommend
punting and disabling exec-shield until a work-around for unexec is at
hand.  Your thoughts?

Take care,

Tim Daly writes:

> Some hints from the Fedora Core mailing list about making programs 
> run under exec-shield.
> 
> Tim
> 
> From: Ingo Molnar
> Subject: Re: Executable memory: further programs that fail
> To: Fedora Development <fedora-devel-list@redhat.com>
> Date: Mon, 24 Nov 2003 05:42:27 -0500 (EST)
> 
> 
> On Sat, 22 Nov 2003, Gerard Milmeister wrote:
> 
> > So is it alright to include in Fedora packages that require exec-shield
> > to be turned off? Should there be a wrapper-script that calls the main
> > executable with 'setarch'?
> 
> all 'third party' apps will run with exec-shield disabled by default.
> 
> _Iff_ a package is recompiled under Fedora (using Fedora's gcc and
> binutils) then you might end up with an app that has exec-shield enabled
> although the app itself for whatever reason has some executability
> assumption.
> 
> In this case there are myriads of ways to make the app work again:
> 
>  - use 'setarch i386 <app>' [or rather, use the 'i386 <app>' shortcut]
> 
>  - add this gcc option to the CFLAGS in the Makefile:
> 
> 	-Wa,--execstack
> 
>  - add the following oneliner anywhere in the app's source in an assembly
>    file to disable exec-shield:
> 
> 	.section .note.GNU-stack, "x", @progbits; .previous
> 
>  - disable exec-shield globally, put "kernel.exec-shield = 0" into 
>    /etc/sysctl.conf.
> 
>  - the preferred solution: fix the app itself to not assume executability
>    of non-executable regions or not hardcode any VM properties such as
>    'mmaps start at 2 GB'. Both assumptions are illegal and these apps will
>    likely break on 64-bit architectures too, so they should be fixed. Note
>    that typically the changes needed are quite simple.
> 
> in any case, please keep reporting apps that need an executable stack, we
> want to track them all down and fix them for good. Fortunately 99.9% of
> the 5000+ binaries in a full Fedora Core 1 install work out of box.
> 
> note that there are also prelink related problems, which we want to know
> about and fix just as much, but which should not be confused with
> exec-shield problems. Sometimes these do get mixed up.
> 
> (in any case, please also read the extensive description provided by
> Roland McGrath's "exec-shield mmap & brk randomization" email.)
> 
> 	Ingo

\start
Date: Mon, 1 Dec 2003 19:21:37 -0500 
From: Bill Page
To: Bertfried Fauser
Subject: Re: AXIOM from cvs does not compile

On Monday, December 01, 2003 11:50 AM Bertfried Fauser
>
> ... Its fixed and --heureka- this solves my TeXmacs
> problem too.

Excellent!

Now if only doing theoretical physics was this easy ...
<grin>

\start
Date: Mon, 1 Dec 2003 07:51:16 -0500
From: Tim Daly
To: Bertfried Fauser
Subject: Re: AXIOM from cvs does not compile

The time has come to remove the AXIOM variable.
And apply Juergen's case patch.
Unfortunately I'll be in Italy this week trying to make the case
for literate programming but I'll move these up on the todo list.
I suppose it would even be useful to publish the current todo list.
I'll put it on the todo list :-)

\start
Date: Tue, 2 Dec 2003 08:25:43 -0500
From: Tim Daly
To: Camm Maguire
Subject: Re: Executable memory: further programs that fail]

Camm,

I believe you can query the exec-shield state by 

  cat /proc/sys/kernel/exec-shield

and probably fail to build if it is set to 1.
The user will have to be root to set it which is not required
otherwise unless they want to do a make install.

I can add it to the Axiom configure script.

I've raised objections to exec-shield on the fedora mailing
list (I'm a developer). The code has only been added in the last
two weeks so now is the time to complain. However the complaints
have been dismissed. I've read all the docs they pointed out to
me and nowhere is it explained how it helps security. It does 
hurt memory management but the basic response is "hey, nobody
REQUIRES memory to be contiguous". It's about like putting chairs
on the football field and claiming that there is nothing in the
rules that says this is illegal. Of course every team at every
game will have to play "around" the random chairs. 

I'm not sure how to fix unexec. It appears that a possible solution
will be to dynamically rebuild memory. One poster claimed that the
"randomized shared library locations" will only happen in a 100Meg
area. A possible strategy is to find the highest allocated byte
in the highest shared library, mark the area below as part of the
image and only allocate above that mark. Unexec could "save" the
shared libraries as part of the image even though they are relinked
again on restart.

In general I'm rather confused about the whole issue. I'll have to
read the unexec code in Emacs and get a real clue.

\start
Date: 02 Dec 2003 15:08:22 +0100
From: Peter Simons
To: list
Subject: Re: Executable memory: further programs that fail]

root  writes:

 > I believe you can query the exec-shield state by 

 >   cat /proc/sys/kernel/exec-shield

The more I read about this, the more I get the impression
that "exec-shield" is something entirely different than the
"ProPolice" system I'm running here.

Just curious, are there any ideas how to fix the issues with
PaX and ProPolice? (Other than disabling them.)

\start
Date: Tue, 2 Dec 2003 09:15:50 -0500
From: Tim Daly
To: Peter Simons
Subject: Re: Executable memory: further programs that fail]

Peter,

ProPolice was mentioned on the Fedora-devel-list but I don't
believe they mentioned a way to disable it. The exec-shield discussion
centered around the random shared library placement issue and, as it 
was off-topic, I let it drop.

exec-shield can be disabled with:

echo 0 >/proc/sys/kernel/exec-shield

\start
Date: 02 Dec 2003 09:53:43 -0500
From: Camm Maguire
To: Tim Daly
Subject: Re: Executable memory: further programs that fail]

Hi Tim!

Tim Daly writes:

> Camm,
> 
> I believe you can query the exec-shield state by 
> 
>   cat /proc/sys/kernel/exec-shield
> 
> and probably fail to build if it is set to 1.
> The user will have to be root to set it which is not required
> otherwise unless they want to do a make install.
> 

OK, I'll probe this file, query its value, and add -Wa,--execstack to
the flags for now.  Will let you know when this is done.

> I can add it to the Axiom configure script.
> 
> I've raised objections to exec-shield on the fedora mailing
> list (I'm a developer). The code has only been added in the last

Thanks for your lisp defense efforts! :-)

> two weeks so now is the time to complain. However the complaints
> have been dismissed. I've read all the docs they pointed out to
> me and nowhere is it explained how it helps security. It does 
> hurt memory management but the basic response is "hey, nobody
> REQUIRES memory to be contiguous". It's about like putting chairs
> on the football field and claiming that there is nothing in the
> rules that says this is illegal. Of course every team at every
> game will have to play "around" the random chairs. 

>From our conversation with Roland, there appears to be one area they
would consider changing, and this would help us a lot -- to forgo brk
randomization, as apart from mmap, malloc, and ld.so randomization.
Roland thought this randomization was a violation of the 'spec' of brk
(I agree) and therefore should not be implemented, rather people
should be encouraged to not use brk in favor of more 'modern' memory
allocation routines.  If I were in your position of influence, I would
lobby for this, and not waste your wind trying to blow down
exec-shield in its entirety.  This simple change would solve virtually
all our problems.  It would however leave a non-randomized allocation
path on the system, which may not be desirable from the point of view
of security.  Just to be clear, we might be able to work around a
randomized brk as long as someone fixes unexec, but it will leave
holes in the heap and diminish the available memory to the user.

There are security gains to be had from this approach in general, and
that is that an attacker cannot rely on a buffer overflow exploit
being in a certain position in memory.  I've never studied the
construction of such exploits, but it would seem this would pose an
insuperable obstacle.

> 
> I'm not sure how to fix unexec. It appears that a possible solution
> will be to dynamically rebuild memory. One poster claimed that the
> "randomized shared library locations" will only happen in a 100Meg
> area. A possible strategy is to find the highest allocated byte
> in the highest shared library, mark the area below as part of the
> image and only allocate above that mark. Unexec could "save" the
> shared libraries as part of the image even though they are relinked
> again on restart.
> 

Luckily emacs seems to have sufficient importance that someone else
will likely do this for us.  I've been treating unexec as a black box
myself. 

Take care,

> In general I'm rather confused about the whole issue. I'll have to
> read the unexec code in Emacs and get a real clue.

\start
Date: 02 Dec 2003 10:22:29 -0500
From: Camm Maguire
To: Tim Daly
Subject: Re: Executable memory: further programs that fail]

Greetings!  OK, the -Wa,--execstack changes are in cvs, both HEAD and
2.6.1 as I figure you might want a stable series gcl on fedora.  These
options can be added explicitly whether or not the proc file exists
and contains non-zero by setting the CFLAGS environment variable when
running configure, i.e

CFLAGS="-Wa,--execstack" ./configure ....

This will work on your existing version provided you don't specify
--enable-debug.  It will work even with this option in the configure
version just checked into cvs.

\start
Date: 02 Dec 2003 10:26:11 -0500
From: Camm Maguire
To: Peter Simons
Subject: [Re: Executable memory: further programs that fail]

Greetings!

Peter Simons writes:

> root  writes:
> 
>  > I believe you can query the exec-shield state by 
> 
>  >   cat /proc/sys/kernel/exec-shield
> 
> The more I read about this, the more I get the impression
> that "exec-shield" is something entirely different than the
> "ProPolice" system I'm running here.
> 
> Just curious, are there any ideas how to fix the issues with
> PaX and ProPolice? (Other than disabling them.)
> 

While I do not have time right now to search for a solution myself,
if you can discover a simple one and explain it clearly to me, I'll
try to get it into GCL.

\start
Date: Tue, 2 Dec 2003 10:54:00 -0500
From: Tim Daly
To: Camm Maguire
Subject: Re: Executable memory: further programs that fail]

Please remind me again of the correct cvs flags to check out the 2.6.1 latest.

\start
Date: Tue, 2 Dec 2003 10:53:09 -0500
From: Tim Daly
To: Camm Maguire
Subject: Re: Executable memory: further programs that fail]

>From our conversation with Roland, there appears to be one area they
>would consider changing, and this would help us a lot -- to forgo brk
>randomization, as apart from mmap, malloc, and ld.so randomization.
>Roland thought this randomization was a violation of the 'spec' of brk
>(I agree) and therefore should not be implemented, rather people
>should be encouraged to not use brk in favor of more 'modern' memory
>allocation routines.  If I were in your position of influence, I would
>lobby for this, and not waste your wind trying to blow down
>exec-shield in its entirety.  This simple change would solve virtually
>all our problems.  It would however leave a non-randomized allocation
>path on the system, which may not be desirable from the point of view
>of security.  Just to be clear, we might be able to work around a
>randomized brk as long as someone fixes unexec, but it will leave
>holes in the heap and diminish the available memory to the user.

I've only been attacking the brk randomization issue, not all of
exec-shield. The claim is that brk randomization will "only" allocate
within 100M of memory. (We elders-of-the-field consider 1M to be a
significant amount of memory :-) ) Roland appears to be unmoved by
the discussion so I suppose that we may have to fall back on static
linking as the only acceptable solution. 

I forget why static linking failed but I'll have to get up to speed
on that again. I'll be at a workshop in italy until next tuesday so 
nothing is going to happen until then. When I get back I'll figure
out how to statically link Axiom so the issue goes away.

>There are security gains to be had from this approach in general, and
>that is that an attacker cannot rely on a buffer overflow exploit
>being in a certain position in memory.  I've never studied the
>construction of such exploits, but it would seem this would pose an
>insuperable obstacle.

The basic idea is that you find an exploit in library code, find out
where the dynamic library gets loaded, put a bogus "return address"
onto the stack which branches to the library code and run the exploit.
Since you don't know where the dynamic library will get loaded you
can't hard-code a branch address onto the stack. Of course if you'll
let me choose the execution location (because I have compromised your
stack) I pretty much own the machine. 

Your fixed code probably does a call to the dynamic library anyway 
otherwise it wouldn't have linked in the first place. From your
fixed code I can figure out the branch point your code will use,
compute the offset I want to exploit and push THAT address onto
the stack. Randomizing dynamic library locations is just a trivial
piece of fluff as far as security goes. However it is a huge 
performance hit as far as memory management goes. The tradeoff isn't
worth the effort. I have yet to see a convincing argument otherwise.

>Luckily emacs seems to have sufficient importance that someone else
>will likely do this for us.  I've been treating unexec as a black box
>myself. 

Yeah, somebody will figure out unexec. It may even be me (though I
doubt it) :-)

\start
Date: 02 Dec 2003 11:19:26 -0500
From: Camm Maguire
To: Tim Daly
Subject: Re: Executable memory: further programs that fail]

Greetings!

cvs -d:pserver:anoncvs@subversions.gnu.org:/cvsroot/gcl co -r
Version_2_6_1 -d gcl-2.6.1 gcl

Tim Daly writes:

> Please remind me again of the correct cvs flags to check out the 2.6.1 latest.

\start
Date: 02 Dec 2003 11:27:17 -0500
From: Camm Maguire
To: Tim Daly
Subject: Re: Executable memory: further programs that fail]

Greetings!

Tim Daly writes:

> I've only been attacking the brk randomization issue, not all of
> exec-shield. The claim is that brk randomization will "only" allocate
> within 100M of memory. (We elders-of-the-field consider 1M to be a
> significant amount of memory :-) ) Roland appears to be unmoved by

Just wanted to point out here that a randomization of this magnitude,
even if gcl can work around it, could potentially result in a loss of
100M in usable memory on each save-system.  I think axiom currently
uses about 4 of these.  The default memory maximum for GCL is only
128M.  Count me among the elders :-).

> the discussion so I suppose that we may have to fall back on static
> linking as the only acceptable solution. 
> 

If you recall, static linking was broken even without exec-shield.  I
think if brk randomization remains at the above mentioned range, we
would probably fall back on Roland's linker script solution, if I
understand correctly. 

> The basic idea is that you find an exploit in library code, find out
> where the dynamic library gets loaded, put a bogus "return address"
> onto the stack which branches to the library code and run the exploit.
> Since you don't know where the dynamic library will get loaded you
> can't hard-code a branch address onto the stack. Of course if you'll
> let me choose the execution location (because I have compromised your
> stack) I pretty much own the machine. 
> 
> Your fixed code probably does a call to the dynamic library anyway 
> otherwise it wouldn't have linked in the first place. From your
> fixed code I can figure out the branch point your code will use,
> compute the offset I want to exploit and push THAT address onto
> the stack. Randomizing dynamic library locations is just a trivial
> piece of fluff as far as security goes. However it is a huge 
> performance hit as far as memory management goes. The tradeoff isn't
> worth the effort. I have yet to see a convincing argument otherwise.
> 

OK, your understanding is considerably deeper than mine on this
issue.  Even granting some security benefit, though, I don't see why
the range can't be quite small and still be effective.  How long would
it take someone to guess a randomly generated address in even a 1k
range? 

> >Luckily emacs seems to have sufficient importance that someone else
> >will likely do this for us.  I've been treating unexec as a black box
> >myself. 
> 
> Yeah, somebody will figure out unexec. It may even be me (though I
> doubt it) :-)
> 

\start
Date: Tue, 2 Dec 2003 11:45:55 -0500
From: Tim Daly
To: Camm Maguire
Subject: Re: Executable memory: further programs that fail]

>Just wanted to point out here that a randomization of this magnitude,
>even if gcl can work around it, could potentially result in a loss of
>100M in usable memory on each save-system.  I think axiom currently
>uses about 4 of these.  The default memory maximum for GCL is only
>128M.  Count me among the elders :-).

Is there any way to get GCL to use all available memory? Fedora will
let you address 4G so Roland doesn't understand why I cry about 100M.

>If you recall, static linking was broken even without exec-shield.  I
>think if brk randomization remains at the above mentioned range, we
>would probably fall back on Roland's linker script solution, if I
>understand correctly. 

I don't understand Roland's linker solution.

>OK, your understanding is considerably deeper than mine on this
>issue.  Even granting some security benefit, though, I don't see why
>the range can't be quite small and still be effective.  How long would
>it take someone to guess a randomly generated address in even a 1k
>range? 

The issue is the same at 1k vs 100M but the latter sounds so much more
secure :-)

\start
Date: Tue, 2 Dec 2003 11:49:34 -0500
From: Tim Daly
To: Camm Maguire
Subject: Re: Executable memory: further programs that fail]

I just got done talking to my son. He is working on a graphics library
that runs in common lisp (was CMUCL, now SBCL). It uses CLX. Does GCL
support CLX? I tried to coerce...umm, convince him to use GCL but he
claims that the ANSI loop macro fails. 

If he gets the graphics library up to snuff we can import the graphics
as lisp code rather than C code. I'm building the C graphics code now
and it is painful.

\start
Date: 02 Dec 2003 12:11:26 -0500
From: Camm Maguire
To: Tim Daly
Subject: Re: Executable memory: further programs that fail]

Greetings!

Tim Daly writes:

> I just got done talking to my son. He is working on a graphics library
> that runs in common lisp (was CMUCL, now SBCL). It uses CLX. Does GCL
> support CLX? I tried to coerce...umm, convince him to use GCL but he

Last I heard, CLX compiles on GCL out of the box.  Would appreciate
reports. 

> claims that the ANSI loop macro fails. 
> 

Is he using our ANSI build?  I'd appreciate a report here too.  We
have an ansi test suite courtesy of Paul Dietz, which gives the loop
macro a heavy workout.  Last I checked we were as good as cmucl in
this regard.  In the Debian packaged gcl, the test suite results can
be found in /usr/share/doc/{gcl,gclcvs}/test_results.gz.  If one wants
ANSI, our gclcvs branch has made some significant improvements over
the stable branch.  Just to let you know, though, axiom cannot use the
ansi build until the in-package issue is resolved, at least without
being patched, i.e. externally.  Our traditional non-ansi image is
used to build axiom for Debian, as it has the old in-package
behavior. 

> If he gets the graphics library up to snuff we can import the graphics
> as lisp code rather than C code. I'm building the C graphics code now
> and it is painful.
> 

Is this gcl-gtk?  We've imported that tree into cvs head, but haven't
worked with it much yet.

\start
Date: 02 Dec 2003 12:13:10 -0500
From: Camm Maguire
To: Tim Daly
Subject: Re: Executable memory: further programs that fail]

Greetings!

Tim Daly writes:

> >Just wanted to point out here that a randomization of this magnitude,
> >even if gcl can work around it, could potentially result in a loss of
> >100M in usable memory on each save-system.  I think axiom currently
> >uses about 4 of these.  The default memory maximum for GCL is only
> >128M.  Count me among the elders :-).
> 
> Is there any way to get GCL to use all available memory? Fedora will
> let you address 4G so Roland doesn't understand why I cry about 100M.
> 

Sure, --enable-maxpage should let you go as high as you want.  I can't
imagine there not being serious performance issues, but I could be
mistaken. 

> >If you recall, static linking was broken even without exec-shield.  I
> >think if brk randomization remains at the above mentioned range, we
> >would probably fall back on Roland's linker script solution, if I
> >understand correctly. 
> 
> I don't understand Roland's linker solution.
> 

Me either, but sounded promising :-).

> >OK, your understanding is considerably deeper than mine on this
> >issue.  Even granting some security benefit, though, I don't see why
> >the range can't be quite small and still be effective.  How long would
> >it take someone to guess a randomly generated address in even a 1k
> >range? 
> 
> The issue is the same at 1k vs 100M but the latter sounds so much more
> secure :-)
> 

\start
Date: Tue, 2 Dec 2003 14:42:36 -0500
From: Tim Daly
To: list
Subject: Re: Executable memory: further programs that fail]
Cc: Camm Maguire

Camm,

it appears that the 2-6-1 build has a minor bug. In the final build
line there is a

cat init_pre_gcl.lsp.tmp | sed \

and one of the sed replacement commands reads:

-e "s,@LI-CC@,\"gcc -c -g -O2 -Wa,--execstack -Wall ......
                                 ^
                                bug?

It appears that the gcc command generation line has a comma 
that should not be there.

\start
Date: 02 Dec 2003 15:19:02 -0500
From: Camm Maguire
To: Tim Daly
Subject: Re: Executable memory: further programs that fail]

Thanks!  Should be fixed now.

Take care,

Tim Daly writes:

> Camm,
> 
> it appears that the 2-6-1 build has a minor bug. In the final build
> line there is a
> 
> cat init_pre_gcl.lsp.tmp | sed \
> 
> and one of the sed replacement commands reads:
> 
> -e "s,@LI-CC@,\"gcc -c -g -O2 -Wa,--execstack -Wall ......
>                                  ^
>                                 bug?
> 
> It appears that the gcc command generation line has a comma 
> that should not be there.

\start
Date: Sun, 7 Dec 2003 01:20:09 +0100 (CET)
From: Bertfried Fauser
To: list
Subject: AXIOM / TeXmacs problems (bugs?)

Hi,

I am not quite sure to which mailing list I should send the following. I
am also not sure if some of my findings are bugs or features ... hence I
will provide a few and compile later a longer list which I send after
seeing where to go with.


AXIOM (TeXmacs and cmd-line)
1) The .axiom.input file cannot be red by the command
   (1)-> )read .axiom
   it seem as if AXIOM parses teh filemane and does not recognize such
   filenames which are starting with a '.' (Since this file is loaded
   automatically its not serious)
   WORKAROUND:
   (1)-> )read .axiom.input

2) AXIOM using TeXmacs
  In the cmd-line AXIOM I can define functions using several ines of input
  where lines which are not the last line are ended by ' _'. In TeXmacs
  this confused TeXmacs totally (freezes teh program). I consider this a
  bug, since it schould be possible to come up with multi line command
  (TeXmacs usesd 'shift RETURN' which might be needed to translated
  into ' _'
  WORKAROUND:
  Use emacs to createand input file foo.input and load it into TeXmascs
  via )read foo, however this is not WYSYWIG fashion

QUESTION:
  Should such findings be reportted in teh mailing list or shall I use teh
  bug-report form? I have a couple of errors of the open source AXIOM
  which was reported to work in the NAG version.

\start
Date: Sun, 07 Dec 2003 11:20:31 +0100
From: David Mentre
To: Bertfried Fauser
Subject: Re: AXIOM / TeXmacs problems (bugs?)

Hello Bertfried,

Bertfried Fauser writes:

>   Should such findings be reportted in teh mailing list or shall I use teh
>   bug-report form? I have a couple of errors of the open source AXIOM
>   which was reported to work in the NAG version.

You should use the bug-report form on Savannah. It helps us not forget
issues. You can however send an email afterwards to inform people of the
issues. 

Savannah is currently down so the mailing-list is the best second
choice.

\start
Date: Sun, 7 Dec 2003 12:14:45 +0100 (CET)
From: Joris van der Hoeven
To: Bertfried Fauser
Subject: Re: AXIOM / TeXmacs problems (bugs?)

> 2) AXIOM using TeXmacs
>   In the cmd-line AXIOM I can define functions using several ines of input
>   where lines which are not the last line are ended by ' _'. In TeXmacs
>   this confused TeXmacs totally (freezes teh program). I consider this a
>   bug, since it schould be possible to come up with multi line command
>   (TeXmacs usesd 'shift RETURN' which might be needed to translated
>   into ' _'

This can indeed be done (the interface for GNUplot does such a thing,
for instance). The maintainers of Axiom interface might wish to take
at the GNUplot interface and do the same thing for Axiom.

\start
Date: Sun, 07 Dec 2003 13:07:22 +0100
From: David Mentre
To: list
Subject: )read .axiom issue (was: Re: AXIOM / TeXmacs	problems (bugs?))

Hello,

Bertfried Fauser writes:

> 1) The .axiom.input file cannot be red by the command
>    (1)-> )read .axiom
>    it seem as if AXIOM parses teh filemane and does not recognize such
>    filenames which are starting with a '.' (Since this file is loaded
>    automatically its not serious)
>    WORKAROUND:
>    (1)-> )read .axiom.input

Ok. The code that make the )read command is defined in
src/interp/i-syscmd.boot.pamphlet, lines 1406-..., in the readSpad2Cmd
boot function. At first glance, I cannot see what is going wrong.

Tim, I tried to follow your guidelines to trace this function execution
but the execution of readSpad2Cmd does not stop. Have I missed something?

(1) -> )trace readSpad2Cmd )break
 
   Function traced: readSpad2Cmd 
(1) -> )read .axiom
1<enter readSpad2Cmd : (|.axiom|)
protected-symbol-warn called with (NIL)
Break on entering readSpad2Cmd :
 
   The file .axiom is needed but does not exist.


Back to Bertfried issue: it appears that the call to FINDFILE returns
NIL when called with ".axiom" and the found file with ".axiom.input". 

(1) -> )trace _$FINDFILE
 
   Function traced: $FINDFILE 
(1) -> )read .axiom
1<enter $FINDFILE : #p".axiom"\("input" "INPUT" "boot" "BOOT" "lisp" "LISP")
1>exit  $FINDFILE : NIL
 
   The file .axiom is needed but does not exist.
(1) -> )read .axiom.input
1<enter $FINDFILE : #p".axiom.input"\("input" "INPUT" "boot" "BOOT" "lisp" "LISP")
1>exit  $FINDFILE : "/home/david/.axiom.input"


Is it a Common Lisp issue?

\start
Date: Mon, 08 Dec 2003 20:00:37 +0100
From: David Mentre
To: Bill Page
Subject: Re: [Axiom-mail] RE: AXIOM / TeXmacs problems (bugs?)

[ Continuing on axiom-developer... ]

Hello,

Bill Page writes:

> There are differences however even in the semantics when
> you write library routines that are intended to be compiled.
> For example you must provide much more explicit type
> information since you can no longer depend on the interpreter's
> heuristics for type inference. I am not sure how ambitious
> one should be in attempting to resolve such differences.

Another approach might to use only a compiler, the "interpreter" just
feeding input to the compiler and displaying its output. This is the
approach followed in the Objective Caml language. 

Such a design would raise the issue of having a good type inference
algorithm that would allow (1) a light syntax (an absolut necessity for
the interactive mode) and (2) would be, at the same time, a unique, well
defined and correct type inference. From the few pages I have read in
the Jenks & Sutor book, the "label" type inference (or "record" type
inference) as found in the Objective Caml language and used for the
typing of objects (as in object-oriented) might be sufficient. But
clearly more work would be necessary to have a light syntax in
interactive mode[1].

Yours,
d.

[1] In short, in ML like languages, the type of objects is deduced from
    the name of the operator (e.g. '+' has '(INT, INT) -> INT'
    signature, using Axiom notation). In Axiom, it appears to me that
    typing is more object-oriented like, in the same way as SmallTalk or
    C++ with overloading: the type of objects determines the type of
    operation to apply. One might envision a typing system where you
    have several operators ('+int' for (INT, INT) -> INT, '+poly' for
    (POLY, POLY) -> POLY) (thus you can use ML-like type inference) but
    hide those several operators with some "hints" from the typing of
    previous objects in interactive mode (e.g. x is a POLY, so the '+'
    in "x+3" means "use the '+poly'"). I don't know yet if this idea is
    just smoke in my brain or a real good idea. :)

\start
Date: Tue, 9 Dec 2003 11:25:09 -0500
From: Bill Page
To: David Mentre
Subject: Axiom interactive input syntax (was: [Axiom-mail] RE: AXIOM / TeXmacs problems(bugs?))

On Monday, December 08, 2003 2:01 PM David Mentre
wrote:

> 
> Another approach might to use only a compiler, the 
> "interpreter" just feeding input to the compiler and 
> displaying its output. This is the approach followed in
> the Objective Caml language.

There is also a mode like this in Aldor, the Axiom library
compiler. But compared to the Axiom interpreter it seems
very awkward. Maybe a TeXmacs - Aldor interface would help.
I wonder if anyone has done this yet? I imagine that some
people might already use emacs with Aldor in this way.
 
> 
> Such a design would raise the issue of having a good type 
> inference algorithm that would allow (1) a light syntax
> (an absolute necessity for the interactive mode) and
> (2) would be, at the same time, a unique, well defined and
> correct type inference.

That is certainly the hard part! <grin>

> From the few pages I have read in the Jenks & Sutor book,
> the "label" type inference (or "record" type inference) as
> found in the Objective Caml language and used for the typing
> of objects (as in object-oriented) might be sufficient. But
> clearly more work would be necessary to have a light syntax
> in interactive mode[1].

That is interesting but compared to what we were discussing
originally about unifying the Axiom interactive input language
syntax with the compiler language syntax - this is what I
would call *very* ambitious.

What I had in mind was much simpler. I thought it might be
useful to modify the TeXmacs/Axiom (tm_axiom) interface
program so that it stores what it gets from TeXmacs (when
you press Enter) into a temporary text file, say
/tmp/texmacs.input, interpreting shift-Enter as new lines,
and then calls Axiom with

 )read /tmp/texmacs.input

to execute the commands. This would allow the usual indented
block structure style to be used rather than the ( ) block
structure. And the syntax would then be closer to the
compiler syntax and also essentially identical to the code
that is displayed in the Jenks & Sutor book.

This change is simple so when I have a moment, I will try
it. But I am not sure that it will always work nor whether
it is the best approach.

> 
> Yours,
> d.
> 
> [1] In short, in ML like languages, the type of objects is 
> deduced from the name of the operator (e.g. '+' has
> '(INT, INT) -> INT' signature, using Axiom notation). In
> Axiom, it appears to me that typing is more object-oriented
> like, in the same way as SmallTalk or C++ with overloading:
> the type of objects determines the type of operation to apply.

Yes, that is exactly true. Notice that in interactive Axiom
when we define

  f(x) == x+x

we have not really defined a function yet. It is only a kind
of template for a class of functions. (I am not sure of the
correct technical Axiom term of this.) It is only when I write

  f(2)

that Axiom compiles a function of type

  Integer -> Integer

It is during compilation that + is identified as +$Integer,
although I could have been more specific, if I so choose,
when I wrote the function "template" and included more type
information - the way I would have to if I was writing and
compiling a library routine.

Then when I write

  m := matrix [[1.0,2.0],[3.0,4.0]]
  f(m)

Axiom compiles another function of type

  Matrix Float -> Matrix Float

etc. So even in an online session we might have several
different functions associated with the overloaded name f.

> One might envision a typing system where you have several
> operators ('+int' for (INT, INT) -> INT, '+poly' for
> (POLY, POLY) -> POLY) (thus you can use ML-like type 
> inference) but hide those several operators with some
> "hints" from the typing of previous objects in interactive
> mode (e.g. x is a POLY, so the '+' in "x+3" means
> "use the '+poly'"). I don't know yet if this idea is
> just smoke in my brain or a real good idea. :)

Not at all. That is (more or less) exactly how Axiom works
now. See "package calling" in Jenks and Sutor, page 83.

\start
Date: Tue, 9 Dec 2003 14:32:12 -0500
From: Tim Daly
To: Bill Page
Subject: Re: [Axiom-math] Bourbaki syndrome, Axiom and OpenMath (was: [om] DefMP elements)

Bill,

The openmath library is in the zips subdirectory (OMCv1.4a.tgz) which
I believe was the version that was used with Axiom.

I spoke to Carlo Traverso who was involved with openmath (I've been
away on a business trip to Italy to talk to Carlo). We'll clearly 
have further disucssions on this subject.

\start
Date: Wed, 10 Dec 2003 08:41:40 +0100
From: David Mentre
To: Bill Page
Subject: Re: Axiom interactive input syntax

Bill Page writes:

> On Monday, December 08, 2003 2:01 PM David Mentre
> wrote:
>
>> (2) would be, at the same time, a unique, well defined and
>> correct type inference.
>
> That is certainly the hard part! <grin>

A hard part, but *maybe* not the most difficult one (if record-like type
inference suits Axiom needs).

> That is interesting but compared to what we were discussing
> originally about unifying the Axiom interactive input language
> syntax with the compiler language syntax - this is what I
> would call *very* ambitious.

Yes. :) Never let a computer scientist try to make some science. :)

> What I had in mind was much simpler. I thought it might be
> useful to modify the TeXmacs/Axiom (tm_axiom) interface
> program so that it stores what it gets from TeXmacs (when
> you press Enter) into a temporary text file, say
> /tmp/texmacs.input, interpreting shift-Enter as new lines,
> and then calls Axiom with
>
>  )read /tmp/texmacs.input
>
> to execute the commands. This would allow the usual indented
> block structure style to be used rather than the ( ) block
> structure. And the syntax would then be closer to the
> compiler syntax and also essentially identical to the code
> that is displayed in the Jenks & Sutor book.

Yes, it might be doable. However I wonder if we want to have a different
syntax between TeXmacs and command-line interfaces. Wouldn't it disturb
the user? More burden for the community? And would the example of the
Axiom book be usable?

More generally, Axiom is not perfect. We will have to decide at which
point we make more noise and start its promotion. (I think having the
freely available Axiom book is a requirement) We will probably then have
a more important user community and changes will be more difficult
(due to user habits). I have not predefined answer on such issues. 


>> One might envision a typing system where you have several
>> operators ('+int' for (INT, INT) -> INT, '+poly' for
>> (POLY, POLY) -> POLY) (thus you can use ML-like type 
>> inference) but hide those several operators with some
>> "hints" from the typing of previous objects in interactive
>> mode (e.g. x is a POLY, so the '+' in "x+3" means
>> "use the '+poly'"). I don't know yet if this idea is
>> just smoke in my brain or a real good idea. :)
>
> Not at all. That is (more or less) exactly how Axiom works
> now. See "package calling" in Jenks and Sutor, page 83.

I'm only page 69. :) 

In any case, many thanks for the feedback. 

\start
Date: Wed, 10 Dec 2003 12:15:24 +0100 (CET)
From: Bertfried Fauser
To: David Mentre
Subject: Axiom interactive input syntax
Cc: Bill Page

On Wed, 10 Dec 2003, David Mentre wrote:

Dear Bill and David,

a very interesting discussion. As a prototype of an ignorant user, I would
be satisfied by a clear statement of what kind of input is necessary in
what interface. Just to tp`ype a few ' _' signs in a comand line session
is no trouble. But I hadn't found no hint that the syntax is different.
That was the most confusing point and that is most easily cured I think.


> >> (2) would be, at the same time, a unique, well defined and
> >> correct type inference.
> >
> > That is certainly the hard part! <grin>
>
> A hard part, but *maybe* not the most difficult one (if record-like type
> inference suits Axiom needs).
>
> > That is interesting but compared to what we were discussing
> > originally about unifying the Axiom interactive input language
> > syntax with the compiler language syntax - this is what I
> > would call *very* ambitious.
>
> Yes. :) Never let a computer scientist try to make some science. :)

Why NOT? Or was all of AXIOMS algebra coded by mathematicians? I doubt
it...

However, I do *not* see any possibility to detect automatically the type
of a data structure. Hence AXIOM will for sure fail in some cases to
compile a correct function if no type specification is given.
Eg: A list of intergers could be a list of bases in a Grassmann algebra
*or* a partition *or* a .... if I would like to add these, I need
'+GRASS' and '+PART' and ... and AXIOM cannot decide this. In this sense,
a 'soft' user interface would need to be even inquisitive about the user
and ask interactively about data types if ther eis more than one known
choice for a libary function.
	Even worse if you define functions, which might implement new such
possibilities and the compiler would even need to *understand* the
legality of a proceedure on a certain data structure. That's even a task
difficult for mathematicians.

What would be of utmost help to me would be a very very good graphical
type brouser. Indeed even with the large amount of series one is
tourtored. Furthermore its not so easy to convert types and there should
be when ever possible a cast operator to perform such changes. At least in
such a direction to the more general ttype, hence loosing information.
	Eg a Euclidean ring is also a ring and if Euclidean is not
necesary or even disturbe then it might be dropped. IF later the property
Euclidean of that data is needed, AXIOM is lost, since checking for such a
property might be impossible without further information or user help.
	A brouser could help to keep trak of system wide known typse and
might come up with a dependence structure (like the algebra dependencies?)

\start
Date: Wed, 10 Dec 2003 10:20:50 +0000
From: Nic Doye
To: Bertfried Fauser
Subject: Axiom interactive input syntax
Cc: Bill Page

On Wed, 2003-12-10 at 11:15, Bertfried Fauser wrote:
> What would be of utmost help to me would be a very very good graphical
> type brouser. Indeed even with the large amount of series one is
> tourtored. Furthermore its not so easy to convert types and there should
> be when ever possible a cast operator to perform such changes. At least in
> such a direction to the more general ttype, hence loosing information.
> 	Eg a Euclidean ring is also a ring and if Euclidean is not
> necesary or even disturbe then it might be dropped. IF later the property
> Euclidean of that data is needed, AXIOM is lost, since checking for such a
> property might be impossible without further information or user help.
> 	A brouser could help to keep trak of system wide known typse and
> might come up with a dependence structure (like the algebra dependencies?)

Excuse me if I've got the wrong end of the stick.

I did some work on automatic type changing in the interpreter for my
thesis under James Davenport...

I should have the BOOT code at home on CD somewhere, although I did have
some problems integrating the final version into the interpreter.

You can get the thesis from http://www.nic.uklinux.net/research/phd.ps

nic

(PS. Hi to Mike Dewar and anyone else I knew back then).
-- 
Nic Doye

\start
Date: 10 Dec 2003 10:03:14 -0500
From: Camm Maguire
To: Vadim V. Zhytnikov
Subject: Re: [Gcl-devel] GCL on mingw
Cc: Mike Thomas

Hi Vadim!

Vadim V. Zhytnikov writes:

> Hi!
> 
> Trying to build recent GCL 2.6.1 on mingw I encountered

  ^^^^^^^^^^^^^^^

Great!  I was just going to ask you if you had a mingw development
system to work with given your earlier mingw problem report.

> some strange problem on configure stage.
> The following test fails
> ===========================================================
> echo $ac_n "checking sizeof struct contblock""... $ac_c" 1>&6
> echo "configure:3238: checking sizeof struct contblock" >&5
> if test "$cross_compiling" = yes; then
>    echo Cannot find sizeof struct contblock;exit 1
> else
>    cat > conftest.$ac_ext <<EOF
> #line 3243 "configure"
> #include "confdefs.h"
> #include <stdio.h>
> 	#define EXTER
> 	#include "$MP_INCLUDE"
> 	#include "`pwd`/h/enum.h"
> 	#include "`pwd`/h/object.h"
> 	int main(int argc,char **argv,char **envp) {
> 	FILE *f=fopen("conftest1","w");
> 	fprintf(f,"%u",sizeof(struct contblock));
> 	fclose(f);
> 	return 0;
> 	}
> EOF
> ===========================================================
> Trouble makes are these two lines
> 	#include "`pwd`/h/enum.h"
> 	#include "`pwd`/h/object.h"
> Due to some reason under mingw
> 	#include "/home/vadim/gcl/h/enum.h"
> signals an error: File not found.
> I really don't understand such strange behavior
> since ls /home/vadim/gcl/h/enum.h works fine.
> But maybe we just can replace these two lines by
> 	#include "h/enum.h"
> 	#include "h/object.h"

I'll look into making this change.  Unfortunately, write access to cvs
at savannah is still down.  Please remind me if I forget by the time
it is restored.  

> 
> With such modification I was able to build
> ANSI GCL on mingw.  My goal was to test strange

  ^^^^

Really?  You built pcl?  We definitely need the details here if so, as
Mike has experienced problems getting through this stage, and has had
to use precompiled .c source to ship his binary ansi package.

BTW, I do suspect the problem you report and Mike's build problem
stems from the same source.

> memory-related GCL crashes under mingw.
> I tried various memory allocation tests -
> exactly the same I used on Linux (see e.g.
> atest.lisp in attachment).  In general
> results are practically the same on both
> platforms with one important exception.
> While on Linux I maximally can use 110K pages
> (MAXPAGES=128K) on mingw all attempts
> to allocate more than ~62000 pages
> causes allocation error.  GCL self terminates
> with the message:
> Unrecoverable error: Can't allocate
> 

My suspicion is that the heap is growing into some memory area already
in use for something else, e.g. shared libs.  I think Mike is away at
the moment, but I had previously requested the following from him,
which you may now be able to provide for me:

1) value of the configure determined define DBEGIN
2) on building a gcl with --enable-debug, run under gdb, breaking at
   main, and report the value of 'p sbrk(0)'
3) break at init_lisp, stop at this line:

	if (NULL_OR_ON_C_STACK(&j) == 0
	    || NULL_OR_ON_C_STACK(Cnil) != 0
	    || (((unsigned long )core_end) !=0
	        && NULL_OR_ON_C_STACK(core_end) != 0))
	  { /* check person has correct definition of above */
	    error("NULL_OR_ON_C_STACK macro invalid");
	  }

   and report the values returned by 'p &j', 'p &Cnil_body', and 'p
   core_end'.

4) Try to let me know if the C stack counts up or down.  I.e. break in
   some function with a local variable defined, and print the address
   of that variable, and compare it to the address of a local variable
   defined in a surrounding function (i.e. a parent function).

We have the following somewhat less than robust code currently in
place for MINGW (main.c):

#ifdef _WIN32
unsigned int _dbegin = 0x10100000;
unsigned int _stacktop, _stackbottom;
#endif

#ifdef _WIN32
	  {
	    unsigned int dummy;
	    
	    _stackbottom = (unsigned int ) &dummy;
	    _stacktop    = _stackbottom - 0x10000; // ???

	  }
#endif


So from this, sbrk(0) should begin at around 0x10100000, and the stack
should count down from some unknown (to me at least) address region.
Please try to verify this and fill in the holes.  It would be great to
firm this up, particularly the hardcoded stack area limit.

Then, if Mingw has some analog of ldd or /proc/$pid/maps, please
report their contents/output to me on the image running under gdb.
I.e. 'ldd saved_gcl' and 'cat /proc/(process id of saved_gcl)/maps'. 

I'm assuming the error message you saw was:

	IF_ALLOCATE_ERR error("Can't allocate.  Good-bye!");

(There are a few other error messages beginning with 'Can't
allocate').  If so, my guess is that sbrk has hit a large jump.  We
have another somewhat ad hoc piece of code in place for mingw at
present  (mingw.h):

#define IF_ALLOCATE_ERR \
	if (core_end != sbrk(0))\
         {char * e = sbrk(0); \
	if (e - core_end < 0x10000 ) { \
	  int i; \
	  for (i=page(core_end); i < page(e); i++) { \
	    type_map[i] = t_other; \
	  } \
	  core_end = e; \
	} \
          else  \
        error("Someone allocated my memory!");} \
	if (core_end != (sbrk(PAGESIZE*(n - m))))



The analog for linux (bsd.h):

#define ROUND_UP_SBRK(x)  \
       do {long i; \
	     if ((i = ((long)x & (PAGESIZE - 1)))) \
	       x=sbrk(PAGESIZE - i); } while(0);

#define FIX_RANDOM_SBRK \
do {char *x=sbrk(0); \
  if (core_end != x) \
   { ROUND_UP_SBRK(x); x=sbrk(0);\
     while (core_end < x) \
       { type_map[page(core_end)]= t_other; \
	 core_end = core_end + PAGESIZE;} \
     if (core_end !=x) error("Someone allocated my memory");}} while (0)
 
     
#define IF_ALLOCATE_ERR \
        FIX_RANDOM_SBRK; \
	if (core_end != sbrk(PAGESIZE*(n - m)))


has no prescribed limit of 0x10000.  Mike, where does this come from?

As a bonus, examining this code leads me to suspect that we already
have designed in mechanims to handle non-contiguous sbrk, a la
exec-shield, meaning that it is likely that someone has made sure GCL
would work under an exec-shield like randomized sbrk, barring unexec
problems as earlier discussed.

This doesn't yet address your other post, where there is no "Can't
allocate" error, but gives an important clue, I feel.


Take care,

> At present I don't know why this happens.
> Any ideas?
> 
> 
> 
> -- 
>       Vadim V. Zhytnikov
> 
>        <Vadim V. Zhytnikov>
>       <Vadim V. Zhytnikov>
> (si::allocate-growth 'cons 1 1000 66 33)
> (setq cnt 0)
> (si::gbc-time 0)
> (setq w nil)
> 
> (defun pass ()
>   (progn (setq cnt (1+ cnt))
>     (format t "***** Starting pass #~d" cnt)
>     (time(setq w (cons (make-list 3000000) w)))
>     (room)
>     (format t "***** End of pass #~d" cnt) (terpri)
>     (format t "***** Run time: ~,2F  GC time: ~,2F (~,1F%)" 
>       (/  (get-internal-run-time) 100.0)
>       (/  (si::gbc-time) 100.0)
>       (* (/ (si::gbc-time) (get-internal-run-time)) 100.0))
>     ))

\start
Date: Wed, 10 Dec 2003 11:26:34 -0500
From: Bill Page
To: Nic Doye
Subject: Axiom interactive input syntax

On Wednesday, December 10, 2003 5:21 AM nic

> ... 
> Excuse me if I've got the wrong end of the stick.
>

Not at all, you have come to the right place!
 
> I did some work on automatic type changing in the
> interpreter for my thesis under James Davenport...
> 
> I should have the BOOT code at home on CD somewhere, although 
> I did have some problems integrating the final version into 
> the interpreter.
> 
> You can get the thesis from
>    http://www.nic.uklinux.net/research/phd.ps
> 

Thank you very much indeed for this contribution. I have
so far only just quickly skimmed your thesis but it appears
to me to be *exactly* on the topic that we are discussing.
And it seems "ambitious" in exactly the kind of way that
David Mentre outlined earlier.

I personally am very highly motivated to implement and test
the kind of changes to Axiom that you describe in your thesis.
Now that Axiom is open source and we have a small (but growing)
body of Axiom expertise available here, I think it would be
great to carry out the kind of project that your thesis
initiates.

Tim, can we make some arrangement, e.g. a new CVS development
branch where Nic could upload his modified BOOT code and
we could proceed with a new experimental version of Axiom
without interfering with the ongoing process of releasing
more of the original Axiom code.

\start
Date: Wed, 10 Dec 2003 20:30:20 +0300
From: Vadim V. Zhytnikov
To: Camm Maguire
Subject: Re: [Gcl-devel] GCL on mingw
Cc: Mike Thomas

Camm Maguire ?????:

> Hi Vadim!
> 
> Vadim V. Zhytnikov writes:
> 
> 
>>Hi!
>>
>>Trying to build recent GCL 2.6.1 on mingw I encountered
> 
> 
>   ^^^^^^^^^^^^^^^
> 
> Great!  I was just going to ask you if you had a mingw development
> system to work with given your earlier mingw problem report.
> 

Well, I just followed Mike's readme.mingw instruction.
The key point is additional (si::use-fast-links nil) in
pcl/makefile.  He also apparently recommends --enable-custreloc
but I was able to build  ANSI GCL with and without this option.
On the other hand I did it with some gcl 2.6.1 snapshot not
with very last CVS sources - I have to try it once again.

> 
>>some strange problem on configure stage.
>>The following test fails
>>===========================================================
>>echo $ac_n "checking sizeof struct contblock""... $ac_c" 1>&6
>>echo "configure:3238: checking sizeof struct contblock" >&5
>>if test "$cross_compiling" = yes; then
>>   echo Cannot find sizeof struct contblock;exit 1
>>else
>>   cat > conftest.$ac_ext <<EOF
>>#line 3243 "configure"
>>#include "confdefs.h"
>>#include <stdio.h>
>>	#define EXTER
>>	#include "$MP_INCLUDE"
>>	#include "`pwd`/h/enum.h"
>>	#include "`pwd`/h/object.h"
>>	int main(int argc,char **argv,char **envp) {
>>	FILE *f=fopen("conftest1","w");
>>	fprintf(f,"%u",sizeof(struct contblock));
>>	fclose(f);
>>	return 0;
>>	}
>>EOF
>>===========================================================
>>Trouble makes are these two lines
>>	#include "`pwd`/h/enum.h"
>>	#include "`pwd`/h/object.h"
>>Due to some reason under mingw
>>	#include "/home/vadim/gcl/h/enum.h"
>>signals an error: File not found.
>>I really don't understand such strange behavior
>>since ls /home/vadim/gcl/h/enum.h works fine.
>>But maybe we just can replace these two lines by
>>	#include "h/enum.h"
>>	#include "h/object.h"
> 
> 
> I'll look into making this change.  Unfortunately, write access to cvs
> at savannah is still down.  Please remind me if I forget by the time
> it is restored.  
> 
> 
>>With such modification I was able to build
>>ANSI GCL on mingw.  My goal was to test strange
> 
> 
>   ^^^^
> 
> Really?  You built pcl?  We definitely need the details here if so, as
> Mike has experienced problems getting through this stage, and has had
> to use precompiled .c source to ship his binary ansi package.
> 
> BTW, I do suspect the problem you report and Mike's build problem
> stems from the same source.
> 

See above.

> 
>>memory-related GCL crashes under mingw.
>>I tried various memory allocation tests -
>>exactly the same I used on Linux (see e.g.
>>atest.lisp in attachment).  In general
>>results are practically the same on both
>>platforms with one important exception.
>>While on Linux I maximally can use 110K pages
>>(MAXPAGES=128K) on mingw all attempts
>>to allocate more than ~62000 pages
>>causes allocation error.  GCL self terminates
>>with the message:
>>Unrecoverable error: Can't allocate
>>
> 
> 
> My suspicion is that the heap is growing into some memory area already
> in use for something else, e.g. shared libs.  I think Mike is away at
> the moment, but I had previously requested the following from him,
> which you may now be able to provide for me:
> 
> 1) value of the configure determined define DBEGIN
> 2) on building a gcl with --enable-debug, run under gdb, breaking at
>    main, and report the value of 'p sbrk(0)'
> 3) break at init_lisp, stop at this line:
> 
> 	if (NULL_OR_ON_C_STACK(&j) == 0
> 	    || NULL_OR_ON_C_STACK(Cnil) != 0
> 	    || (((unsigned long )core_end) !=0
> 	        && NULL_OR_ON_C_STACK(core_end) != 0))
> 	  { /* check person has correct definition of above */
> 	    error("NULL_OR_ON_C_STACK macro invalid");
> 	  }
> 
>    and report the values returned by 'p &j', 'p &Cnil_body', and 'p
>    core_end'.
> 
> 4) Try to let me know if the C stack counts up or down.  I.e. break in
>    some function with a local variable defined, and print the address
>    of that variable, and compare it to the address of a local variable
>    defined in a surrounding function (i.e. a parent function).
> 
> We have the following somewhat less than robust code currently in
> place for MINGW (main.c):
> 
> #ifdef _WIN32
> unsigned int _dbegin = 0x10100000;
> unsigned int _stacktop, _stackbottom;
> #endif
> 
> #ifdef _WIN32
> 	  {
> 	    unsigned int dummy;
> 	    
> 	    _stackbottom = (unsigned int ) &dummy;
> 	    _stacktop    = _stackbottom - 0x10000; // ???
> 
> 	  }
> #endif
> 
> 
> So from this, sbrk(0) should begin at around 0x10100000, and the stack
> should count down from some unknown (to me at least) address region.
> Please try to verify this and fill in the holes.  It would be great to
> firm this up, particularly the hardcoded stack area limit.
> 
> Then, if Mingw has some analog of ldd or /proc/$pid/maps, please
> report their contents/output to me on the image running under gdb.
> I.e. 'ldd saved_gcl' and 'cat /proc/(process id of saved_gcl)/maps'. 

As far as I know there is no analog of /proc/../maps on mingw.

> 
> I'm assuming the error message you saw was:
> 
> 	IF_ALLOCATE_ERR error("Can't allocate.  Good-bye!");
>

Right.

> (There are a few other error messages beginning with 'Can't
> allocate').  If so, my guess is that sbrk has hit a large jump.  We
> have another somewhat ad hoc piece of code in place for mingw at
> present  (mingw.h):
> 
> #define IF_ALLOCATE_ERR \
> 	if (core_end != sbrk(0))\
>          {char * e = sbrk(0); \
> 	if (e - core_end < 0x10000 ) { \
> 	  int i; \
> 	  for (i=page(core_end); i < page(e); i++) { \
> 	    type_map[i] = t_other; \
> 	  } \
> 	  core_end = e; \
> 	} \
>           else  \
>         error("Someone allocated my memory!");} \
> 	if (core_end != (sbrk(PAGESIZE*(n - m))))
> 
> 
> 
> The analog for linux (bsd.h):
> 
> #define ROUND_UP_SBRK(x)  \
>        do {long i; \
> 	     if ((i = ((long)x & (PAGESIZE - 1)))) \
> 	       x=sbrk(PAGESIZE - i); } while(0);
> 
> #define FIX_RANDOM_SBRK \
> do {char *x=sbrk(0); \
>   if (core_end != x) \
>    { ROUND_UP_SBRK(x); x=sbrk(0);\
>      while (core_end < x) \
>        { type_map[page(core_end)]= t_other; \
> 	 core_end = core_end + PAGESIZE;} \
>      if (core_end !=x) error("Someone allocated my memory");}} while (0)
>  
>      
> #define IF_ALLOCATE_ERR \
>         FIX_RANDOM_SBRK; \
> 	if (core_end != sbrk(PAGESIZE*(n - m)))
> 
> 
> has no prescribed limit of 0x10000.  Mike, where does this come from?
> 
> As a bonus, examining this code leads me to suspect that we already
> have designed in mechanims to handle non-contiguous sbrk, a la
> exec-shield, meaning that it is likely that someone has made sure GCL
> would work under an exec-shield like randomized sbrk, barring unexec
> problems as earlier discussed.
> 
> This doesn't yet address your other post, where there is no "Can't
> allocate" error, but gives an important clue, I feel.
> 
> 
> Take care,
> 
> 

I'll be able to do tests you suggest this weekend.
At present I just recompiled GCL with 256K maxpages
but nothing changed.

\start
Date: Wed, 10 Dec 2003 13:06:22 -0500
From: Tim Daly
To: Bill Page, Nic Doye, Bertfried Fauser
Subject: Axiom interactive input syntax

I've downloaded the thesis and quickly browsed it.
This is an excellent piece of work. It appears that a "real"
implementation ran aground on the rocks of "getdatabase"
(my code :-( ). I think we should definitely consider implementing
this code, adding the extensions, and playing with the result.

I've inquired about a way to set up a "research" tree for axiom
on savannah. I'm unaware of how to do this using cvs. I'd like to
be able to incant:

cvs -d:pserver:anoncvs@subversions.gnu.org:/projects/axiom co research

does anyone know how to set up a new root?

\start
Date: Wed, 10 Dec 2003 21:22:09 +0100
From: David Mentre
To: Bertfried Fauser
Subject: Axiom interactive input syntax

Hello Bertfried,

Bertfried Fauser writes:

> What would be of utmost help to me would be a very very good graphical
> type brouser.

Could you elaborate? What kind of behaviour would you like to have?

\start
Date: Wed, 10 Dec 2003 22:19:27 +0100
From: David Mentre
To: Nic Doye
Subject: Type coercicion (was: re: Axiom interactive input syntax)

Hello,

Nic Doye writes:

> I did some work on automatic type changing in the interpreter for my
> thesis under James Davenport...

>From what I have understood from the introduction and conclusion, you
have an algorithm to transform any "type"[1] in any other mathematically
compatible "type" (provided the type system follows some mathematical
guidelines). This makes me ask the following naïve and meta question:
how would you use such an algorithm? From user behavior (tim-user uses
variable x with another operator implying a new type context) or with
explicit user information (calls to coerce)? Within a type inference
algorithm? 

Yours,
david

[1] Sorry to probably not use the proper term.

\start
Date: 10 Dec 2003 16:58:45 -0500
From: Camm Maguire
To: Vadim V. Zhytnikov
Subject: Re: [Gcl-devel] GCL on mingw
Cc: Mike Thomas

Greetings!

Vadim V. Zhytnikov writes:

> Camm Maguire ?????:
> 
> > Hi Vadim!
> > Vadim V. Zhytnikov writes:
> >
> >>Hi!
> >>
> >>Trying to build recent GCL 2.6.1 on mingw I encountered
> >   ^^^^^^^^^^^^^^^
> > Great!  I was just going to ask you if you had a mingw development
> > system to work with given your earlier mingw problem report.
> >
> 
> Well, I just followed Mike's readme.mingw instruction.
> The key point is additional (si::use-fast-links nil) in

                              ^^^^^^^^^^^^^^^^^^^^^^^

Thanks for the reminder.  I had forgotten about this.  I'm wondering
if this is the only place where fast-links cause a problem.  If so, it
may simply be in the large array GCL allocates for the purposes of
toggling between fast and slow function pointers coupled with the
memory allocation issues you are observing..  If not, perhaps there is
some alignment issue, or (much worse), some ia64-like retrictions on
function calls via function pointers when the shared library maps
could change across runs.  We should be able to break at the bad
function jump in pcl_braid.c (see earlier thread with the debugging
details if interested), go up one frame, disassemble, and look at the
registers to see if the function address has somehow been corrupted.

> pcl/makefile.  He also apparently recommends --enable-custreloc
> but I was able to build  ANSI GCL with and without this option.

???  Meaning via statsysbfd (the default), or via dlopen?

> On the other hand I did it with some gcl 2.6.1 snapshot not
> with very last CVS sources - I have to try it once again.
> 

It would be nice if you could also verify the ansi build crash when
not turning off fast-links and building without custreloc.

> >
> >>some strange problem on configure stage.
> >>The following test fails
> >>===========================================================
> >>echo $ac_n "checking sizeof struct contblock""... $ac_c" 1>&6
> >>echo "configure:3238: checking sizeof struct contblock" >&5
> >>if test "$cross_compiling" = yes; then
> >>   echo Cannot find sizeof struct contblock;exit 1
> >>else
> >>   cat > conftest.$ac_ext <<EOF
> >>#line 3243 "configure"
> >>#include "confdefs.h"
> >>#include <stdio.h>
> >>	#define EXTER
> >>	#include "$MP_INCLUDE"
> >>	#include "`pwd`/h/enum.h"
> >>	#include "`pwd`/h/object.h"
> >>	int main(int argc,char **argv,char **envp) {
> >>	FILE *f=fopen("conftest1","w");
> >>	fprintf(f,"%u",sizeof(struct contblock));
> >>	fclose(f);
> >>	return 0;
> >>	}
> >>EOF
> >>===========================================================
> >>Trouble makes are these two lines
> >>	#include "`pwd`/h/enum.h"
> >>	#include "`pwd`/h/object.h"
> >>Due to some reason under mingw
> >>	#include "/home/vadim/gcl/h/enum.h"
> >>signals an error: File not found.
> >>I really don't understand such strange behavior
> >>since ls /home/vadim/gcl/h/enum.h works fine.
> >>But maybe we just can replace these two lines by
> >>	#include "h/enum.h"
> >>	#include "h/object.h"
> > I'll look into making this change.  Unfortunately, write access to
> > cvs
> > at savannah is still down.  Please remind me if I forget by the time
> > it is restored.
> >>With such modification I was able to build
> >>ANSI GCL on mingw.  My goal was to test strange
> >   ^^^^
> > Really?  You built pcl?  We definitely need the details here if so,
> > as
> > Mike has experienced problems getting through this stage, and has had
> > to use precompiled .c source to ship his binary ansi package.
> > BTW, I do suspect the problem you report and Mike's build problem
> > stems from the same source.
> >
> 
> See above.
> 
> >
> >>memory-related GCL crashes under mingw.
> >>I tried various memory allocation tests -
> >>exactly the same I used on Linux (see e.g.
> >>atest.lisp in attachment).  In general
> >>results are practically the same on both
> >>platforms with one important exception.
> >>While on Linux I maximally can use 110K pages
> >>(MAXPAGES=128K) on mingw all attempts
> >>to allocate more than ~62000 pages
> >>causes allocation error.  GCL self terminates
> >>with the message:
> >>Unrecoverable error: Can't allocate
> >>
> > My suspicion is that the heap is growing into some memory area
> > already
> > in use for something else, e.g. shared libs.  I think Mike is away at
> > the moment, but I had previously requested the following from him,
> > which you may now be able to provide for me:
> > 1) value of the configure determined define DBEGIN
> > 2) on building a gcl with --enable-debug, run under gdb, breaking at
> >    main, and report the value of 'p sbrk(0)'
> > 3) break at init_lisp, stop at this line:
> > 	if (NULL_OR_ON_C_STACK(&j) == 0
> > 	    || NULL_OR_ON_C_STACK(Cnil) != 0
> > 	    || (((unsigned long )core_end) !=0
> > 	        && NULL_OR_ON_C_STACK(core_end) != 0))
> > 	  { /* check person has correct definition of above */
> > 	    error("NULL_OR_ON_C_STACK macro invalid");
> > 	  }
> >    and report the values returned by 'p &j', 'p &Cnil_body', and 'p
> >    core_end'.
> > 4) Try to let me know if the C stack counts up or down.  I.e. break
> > in
> >    some function with a local variable defined, and print the address
> >    of that variable, and compare it to the address of a local variable
> >    defined in a surrounding function (i.e. a parent function).
> > We have the following somewhat less than robust code currently in
> > place for MINGW (main.c):
> > #ifdef _WIN32
> > unsigned int _dbegin = 0x10100000;
> > unsigned int _stacktop, _stackbottom;
> > #endif
> > #ifdef _WIN32
> > 	  {
> > 	    unsigned int dummy;
> > 	    	    _stackbottom = (unsigned int ) &dummy;
> > 	    _stacktop    = _stackbottom - 0x10000; // ???
> > 	  }
> > #endif
> > So from this, sbrk(0) should begin at around 0x10100000, and the
> > stack
> > should count down from some unknown (to me at least) address region.
> > Please try to verify this and fill in the holes.  It would be great to
> > firm this up, particularly the hardcoded stack area limit.
> > Then, if Mingw has some analog of ldd or /proc/$pid/maps, please
> > report their contents/output to me on the image running under gdb.
> > I.e. 'ldd saved_gcl' and 'cat /proc/(process id of saved_gcl)/maps'.
> 
> As far as I know there is no analog of /proc/../maps on mingw.
> 
> > I'm assuming the error message you saw was:
> > 	IF_ALLOCATE_ERR error("Can't allocate.  Good-bye!");
> >
> 
> Right.
> 
> > (There are a few other error messages beginning with 'Can't
> > allocate').  If so, my guess is that sbrk has hit a large jump.  We
> > have another somewhat ad hoc piece of code in place for mingw at
> > present  (mingw.h):
> > #define IF_ALLOCATE_ERR \
> > 	if (core_end != sbrk(0))\
> >          {char * e = sbrk(0); \
> > 	if (e - core_end < 0x10000 ) { \
> > 	  int i; \
> > 	  for (i=page(core_end); i < page(e); i++) { \
> > 	    type_map[i] = t_other; \
> > 	  } \
> > 	  core_end = e; \
> > 	} \
> >           else  \
> >         error("Someone allocated my memory!");} \
> > 	if (core_end != (sbrk(PAGESIZE*(n - m))))
> > The analog for linux (bsd.h):
> > #define ROUND_UP_SBRK(x)  \
> >        do {long i; \
> > 	     if ((i = ((long)x & (PAGESIZE - 1)))) \
> > 	       x=sbrk(PAGESIZE - i); } while(0);
> > #define FIX_RANDOM_SBRK \
> > do {char *x=sbrk(0); \
> >   if (core_end != x) \
> >    { ROUND_UP_SBRK(x); x=sbrk(0);\
> >      while (core_end < x) \
> >        { type_map[page(core_end)]= t_other; \
> > 	 core_end = core_end + PAGESIZE;} \
> >      if (core_end !=x) error("Someone allocated my memory");}} while (0)
> >       #define IF_ALLOCATE_ERR \
> >         FIX_RANDOM_SBRK; \
> > 	if (core_end != sbrk(PAGESIZE*(n - m)))
> > has no prescribed limit of 0x10000.  Mike, where does this come from?
> > As a bonus, examining this code leads me to suspect that we already
> > have designed in mechanims to handle non-contiguous sbrk, a la
> > exec-shield, meaning that it is likely that someone has made sure GCL
> > would work under an exec-shield like randomized sbrk, barring unexec
> > problems as earlier discussed.
> > This doesn't yet address your other post, where there is no "Can't
> > allocate" error, but gives an important clue, I feel.
> > Take care,
> >
> 
> I'll be able to do tests you suggest this weekend.
> At present I just recompiled GCL with 256K maxpages
> but nothing changed.
> 

Which would be consistent with the theory that something is occupying
memory at a fixed offset above DBEGIN into which the heap would like
to grow.

\start
Date: Thu, 11 Dec 2003 11:50:01 +0100 (CET)
From: Bertfried Fauser
To: David Mentre
Subject: Axiom interactive input syntax

On Wed, 10 Dec 2003, David Mentre wrote:

> > What would be of utmost help to me would be a very very good graphical
> > type brouser.
>
> Could you elaborate? What kind of behaviour would you like to have?

Dear David,

	perhaps I am describing what the AXIOM browser will do nayway.
However, for me AXIOM is still lacking a f`type browser which can
graphically (like a file system tree) show the dependencies of typs.

Eg (fictious I have currently no AXIOM available)

	browse Ring

	Ring
	  |___ Euclidean Ring
          |       |____ Division Ring
	  |___ Artinear Ring
          |...
	 ...

which shows which types are subtypes, (downsearch)

A second way of working should diplay the upbraces of teh type tree
(graph?) so that you see all datastructures which contain the type
Ring, that may be modules, groups, abelian monoids, etc....

This type of knowledge is importand to program AXIOM packages. I spent
currently the most time with AXIOM by searching correct types (in fact the
most general type which allows me to do teh calculation I want to perform)

I havn't looked into this, but I think the algebra dependen`cies will reflect
somehow the type structure, maybe I am wrong.

\start
Date: Thu, 11 Dec 2003 12:30:42 -0500
From: Tim Daly
To: Nic Doye
Subject: Re: phd thesis
Cc: Bill Page, Bertfried Fauser

Nicolas,

We've discussed setting up a "research" branch of axiom so we can
experiment with new ideas without breaking the build for other users.
I expect there will be several "research" branches because I know of
others who want to experiment with axiom. Indeed, axiom's basic
strength is that it is a very good platform for research.

re: fedora core

The build will almost work fine on Fedora (I'm also a fedora developer
and have been campaigning against the change that breaks axiom). In
order to get around the problem you need to become root and type:

echo 0 >/proc/sys/kernel/exec-shield

Once that is done you can correctly build Axiom as a regular user.
In theory you can download Axiom by incanting:

cd (yourpath)
cvs -d:pserver:anoncvs@subversions.gnu.org:/projects/axiom login
cvs -d:pserver:anoncvs@subversions.gnu.org:/projects/axiom co axiom
cd axiom
export AXIOM=(yourpath)/axiom/mnt/linux
make

When prompted for a password just hit enter.

However, the gnu website is down due to an audit.


I've uploaded a very late (possibly not the latest) version of
Axiom to my own website. You can get it and build it by:

cd (yourpath)
visit axiom.tenken.org
download the "20031211 sources"
tar -zxf axiom.20031211.tgz
cd axiom
export AXIOM=(yourpath)/axiom/mnt/linux
make

re: solaris, etc

>I also have access to Red Hat Enterprise Linux 2.1 and 3 boxen as well
>as Solaris 8 if there's any porting help needed. I can also build
>RPMs...

The other boxen are interesting, particularly the solaris 8 version.
I've tried to download solaris86 to do a build but that whole process
failed and I'll have to try again. If you want to try a port you 
should only need to modify the top level makefile but you'll almost
certainly need my help with that.

re: boot code, thesis, etc

> Yes I hope so!
>It's not very well written as I was more of a Mathematician than a coder
>at the time and I couldn't get the final version to integrate into the
>interpreter (although a previous, lost version had integrated
>perfectly).

not to worry. the code that lives in axiom was never expected to be
released to public view. in fact a lot of the interpreter code is mine
and it is both bad and undocumented. 

>Yes. I'm  pro-open source (I prefer "Free" software, but open source is
>a good compromise).

umm, yeah, we've had endless discussions about this and all discussion
is now "banned" from the axiom-developer mailing list. I've created an
axiom-legal mailing list just for that purpose. I have email from 
Stallman "blessing" the work as "ok". I had no choice about the
modified BSD license so you'll have to flame Mike Dewar about it.

(On the savannah.nongnu.org/projects/axiom website you can sign up
to join the various mailing lists)

>> 3) would you be willing to allow a modified version of your thesis to
>>    be used in a "literate program" style to document that code?

>Yes. Of course!

I (hope, presume) your thesis is in latex. I'd hate to have to retype it.
Literate programs are basically latex code with 2 additional "tags".
You write a document:

\documentclass{...
\begin{document}
\chapter{...
.....

Then you use a chunk definition tag which begins with '<<', contains
any string, and ends with '>>='. Text starting after this tag is
quoted, collected up, and put in a hash table. The chunk ends when
it finds an '@' in column 1. Thus:

\section
lots of good comments
<<any name>>=
  your code goes here
@
\section

Later you can reference the chunk by referencing the name thus:

<<any name>>

There is a "default" chunk name which will get expanded automatically
called '<<*>>=' thus:

<<*>>=
<<first chunk>>
<<next chunk>>
<<any name>>
@

\end{document}

So, suppose you save the pile above as a "literate program".
I refer to them as "pamphlet" files. Lets say you save it as
"foo.pamphlet". You now need 2 tools, one to extract the 
.tex file (noweave) and one to extract the .lisp file (notangle) thus:

noweave  foo.pamphlet >foo.tex
notangle foo.pamphlet >foo.lisp (or .c or .boot, etc)

Thus the pamphlet file contains both the code and the documentation.
The idea is to take the boot code and your thesis, mix the two,
write the intermediate explanations that go from the theory to
the code chunks, and create a pamphlet file (or many pamphlet files)
that document what the code does and why.

You'll be unique as everyone eventually asks how axiom does
coercion and there has never been a good answer to that question
before.

I'm willing to do all of the dog work on making the pamphlet files
if you'll send me the code and the tex files. Once they are mixed
I can build it into the build process and we can work together on
improving the form and content of the pamphlets.

\start
Date: Thu, 11 Dec 2003 21:48:56 +0100
From: David Mentre
To: Bertfried Fauser
Subject: Re: Axiom interactive input syntax

Bertfried Fauser writes:

> 	browse Ring
>
> 	Ring
> 	  |___ Euclidean Ring
>           |       |____ Division Ring
> 	  |___ Artinear Ring
>           |...
> 	 ...
>
> which shows which types are subtypes, (downsearch)
>
> A second way of working should diplay the upbraces of teh type tree
> (graph?) so that you see all datastructures which contain the type
> Ring, that may be modules, groups, abelian monoids, etc....

Thank you Bertfried, it's more clearer now.

\start
Date: Thu, 11 Dec 2003 22:15:05 +0100
From: David Mentre
To: Tim Daly
Subject: GNU Arch (was: Re: [Axiom-math] Re: phd thesis)

Hello Tim,

Tim Daly writes:

> I expect there will be several "research" branches because I know of
> others who want to experiment with axiom. Indeed, axiom's basic
> strength is that it is a very good platform for research.

By the way and for information, GNU Arch is a version control system
(cvs-like) specifically designed to allow separate trees to merge in a
bazaar style. In another words, Arch would match the needs for several
individual experimental research Axioms. I'm not advocating it since
I've never used it. But a TeXmacs developer recently made its promotion
on texmacs-dev mailing list.

\start
Date: Thu, 11 Dec 2003 19:25:26 -0500
From: Bill Page
To: Nic Doye
Subject: RE: phd thesis

On Thursday, December 11, 2003 12:31 PM Tim Daly
> ... 
> I've uploaded a very late (possibly not the latest) version of
> Axiom to my own website. You can get it and build it by:
> 
> cd (yourpath)
> visit axiom.tenken.org
> download the "20031211 sources"
> tar -zxf axiom.20031211.tgz
> cd axiom
> export AXIOM=(yourpath)/axiom/mnt/linux
> make
>

That's actually (just a typo):

  http://axiom.tenkan.org/

> re: solaris, etc
> 
> >I also have access to Red Hat Enterprise Linux 2.1 and 3 
> >boxen as well as Solaris 8 if there's any porting help
> >needed. I can also build RPMs...
> 
> The other boxen are interesting, particularly the solaris 8
> version. I've tried to download solaris86 to do a build but
> that whole process failed and I'll have to try again. If you
> want to try a port you should only need to modify the top level
> makefile but you'll almost certainly need my help with that.
>

Tim, I have an accessible Solaris 8 (on sparc) machine. I will
also try a build when I get a few spare minutes.

\start
Date: Thu, 11 Dec 2003 20:09:20 -0500
From: Bill Page
To: Tim Daly
Subject: Access to Savannah (was RE: phd thesis)

On Thursday, December 11, 2003 12:31 PM Tim Daly

> ... 
> re: fedora core
> 
> The build will almost work fine on Fedora (I'm also a fedora developer
> and have been campaigning against the change that breaks axiom). In
> order to get around the problem you need to become root and type:
> 
> echo 0 >/proc/sys/kernel/exec-shield
> 
> Once that is done you can correctly build Axiom as a regular user.
> In theory you can download Axiom by incanting:
> 
> cd (yourpath)
> cvs -d:pserver:anoncvs@subversions.gnu.org:/projects/axiom login
> cvs -d:pserver:anoncvs@subversions.gnu.org:/projects/axiom co axiom
> cd axiom
> export AXIOM=(yourpath)/axiom/mnt/linux
> make
> 
> When prompted for a password just hit enter.
> 
> However, the gnu website is down due to an audit.
>

The (unaudited) Savannah cvs files are now available at 

  http://savannah.gnu.org/statement.html

where you will find the following instructions:

---------

For example, to check out the CVS module "axiom" in the Savannah
project of the same name, as it existed in the Savannah CVS tree
when we brought the system down, you would need to use the following
commands:

  export CVS_RSH="ssh"
  cvs -d:ext:anoncvs@subversions.gnu.org:/cvs-latest/axiom co axiom

If you already have a subversions.gnu.org entry in your ~/.ssh/config
file with a "Protocol 1" line, you will need to change it to "Protocol
2".

---------

I just tested this and it works for me.

\start
Date: Fri, 12 Dec 2003 17:07:12 +0300
From: Vadim V. Zhytnikov
To: Camm Maguire
Subject: Re: [Gcl-devel] GCL on mingw
Cc: Mike Thomas

Camm Maguire ?????:

> Greetings!
> 
> Vadim V. Zhytnikov writes:
> 
> 
>>Camm Maguire ?????:
>>
>>
>>>Hi Vadim!
>>>Vadim V. Zhytnikov writes:
>>>
>>>
>>>>Hi!
>>>>
>>>>Trying to build recent GCL 2.6.1 on mingw I encountered
>>>
>>>  ^^^^^^^^^^^^^^^
>>>Great!  I was just going to ask you if you had a mingw development
>>>system to work with given your earlier mingw problem report.
>>>
>>
>>Well, I just followed Mike's readme.mingw instruction.
>>The key point is additional (si::use-fast-links nil) in
> 
> 
>                               ^^^^^^^^^^^^^^^^^^^^^^^
> 
> Thanks for the reminder.  I had forgotten about this.  I'm wondering
> if this is the only place where fast-links cause a problem.  If so, it
> may simply be in the large array GCL allocates for the purposes of
> toggling between fast and slow function pointers coupled with the
> memory allocation issues you are observing..  If not, perhaps there is
> some alignment issue, or (much worse), some ia64-like retrictions on
> function calls via function pointers when the shared library maps
> could change across runs.  We should be able to break at the bad
> function jump in pcl_braid.c (see earlier thread with the debugging
> details if interested), go up one frame, disassemble, and look at the
> registers to see if the function address has somehow been corrupted.
> 
> 
>>pcl/makefile.  He also apparently recommends --enable-custreloc
>>but I was able to build  ANSI GCL with and without this option.
> 
> 
> ???  Meaning via statsysbfd (the default), or via dlopen?
> 
> 
>>On the other hand I did it with some gcl 2.6.1 snapshot not
>>with very last CVS sources - I have to try it once again.
>>
> 
> 
> It would be nice if you could also verify the ansi build crash when
> not turning off fast-links and building without custreloc.
> 

I've build latest ANSI GCL (Dec 03, 2003) without custreloc
(all default options except --enable-ansi).  CONS allocation test
fails at the very beginning of 8th pass during RB GBC with
the message
	Can't allocate.  Good-bye!
This is a bit different from traditional GCL. I think that
this difference is purely due to different initial
memory layouts of traditional and ansi images.

\start
Date: 12 Dec 2003 10:59:22 -0500
From: Camm Maguire
To: Vadim V. Zhytnikov
Subject: Re: [Gcl-devel] GCL on mingw
Cc: Mike Thomas

Greetings!

Vadim V. Zhytnikov writes:

> Camm Maguire ?????:
> 
> > Greetings!
> > Vadim V. Zhytnikov writes:
> >
> >>Camm Maguire ?????:
> >>
> >>
> >>>Hi Vadim!
> >>>Vadim V. Zhytnikov writes:
> >>>
> >>>
> >>>>Hi!
> >>>>
> >>>>Trying to build recent GCL 2.6.1 on mingw I encountered
> >>>
> >>>  ^^^^^^^^^^^^^^^
> >>>Great!  I was just going to ask you if you had a mingw development
> >>>system to work with given your earlier mingw problem report.
> >>>
> >>
> >>Well, I just followed Mike's readme.mingw instruction.
> >>The key point is additional (si::use-fast-links nil) in
> >                               ^^^^^^^^^^^^^^^^^^^^^^^
> > Thanks for the reminder.  I had forgotten about this.  I'm wondering
> > if this is the only place where fast-links cause a problem.  If so, it
> > may simply be in the large array GCL allocates for the purposes of
> > toggling between fast and slow function pointers coupled with the
> > memory allocation issues you are observing..  If not, perhaps there is
> > some alignment issue, or (much worse), some ia64-like retrictions on
> > function calls via function pointers when the shared library maps
> > could change across runs.  We should be able to break at the bad
> > function jump in pcl_braid.c (see earlier thread with the debugging
> > details if interested), go up one frame, disassemble, and look at the
> > registers to see if the function address has somehow been corrupted.
> >
> >>pcl/makefile.  He also apparently recommends --enable-custreloc
> >>but I was able to build  ANSI GCL with and without this option.
> > ???  Meaning via statsysbfd (the default), or via dlopen?
> >
> >>On the other hand I did it with some gcl 2.6.1 snapshot not
> >>with very last CVS sources - I have to try it once again.
> >>
> > It would be nice if you could also verify the ansi build crash when
> > not turning off fast-links and building without custreloc.
> >
> 
> I've build latest ANSI GCL (Dec 03, 2003) without custreloc
> (all default options except --enable-ansi).  CONS allocation test

Just checked, and the configure default for mingw is cust-reloc.  I'm
imagining that --disable-custreloc --enable-locbfd will fail.

> fails at the very beginning of 8th pass during RB GBC with
> the message
> 	Can't allocate.  Good-bye!

Sorry for the confusing request.  I wanted to see if you can reproduce
the crash when building pcl from source as part of the ansi build
process when *not* turning off fast-links.  I'm inferring that
cust-reloc must be used on mingw, so I imagine that you will easily
reproduce Mike's reported problem.   Still, it would be helpful to
reproduce this in a directory with --enable-debug turned on so we can
see what the problem is (i.e. whether it is memory related or not.)

> This is a bit different from traditional GCL. I think that
> this difference is purely due to different initial
> memory layouts of traditional and ansi images.

\start
Date: Fri, 12 Dec 2003 18:09:50 +0000
From: Nic Doye
To: Tim Daly
Subject: Re: phd thesis
Cc: Bill Page, Bertfried Fauser

On Thu, 2003-12-11 at 17:30, Tim Daly wrote:

I've located all my source code. I've now got to understand it and work
out which was the more recent version. ;-)

\start
Date: Mon, 15 Dec 2003 11:53:56 +0000
From: Nic Doye
To: David Mentre
Subject: Re: Type coercicion (was: re: Axiom interactive input syntax)

On Wed, 2003-12-10 at 21:19, David Mentre wrote:
> Hello,
> 
> Nic Doye writes:
> 
> > I did some work on automatic type changing in the interpreter for my
> > thesis under James Davenport...
> 
> >From what I have understood from the introduction and conclusion, you
> have an algorithm to transform any "type"[1] in any other mathematically
> compatible "type" (provided the type system follows some mathematical
> guidelines). This makes me ask the following na=EFve and meta question:
> how would you use such an algorithm? From user behavior (tim-user uses
> variable x with another operator implying a new type context) or with
> explicit user information (calls to coerce)? Within a type inference
> algorithm? 

It "should" be invisible to the user (ie. during an internal call to
coerceInteractive() ).

The standard case being that Q[x] has a natural injection into Z(x)
( UPoly Frac Int -> Frac UPoly Int ).

So if a :=3D ( ( 1/2 ) * x**2 + x - 1/3 ) (Upoly Frac Int) we are free to
add a member of Z, Q, Z(x) to it and will "work" without the user having
to call coerce.

That's not to say that it shouldn't be callable by the user, somehow.

It should also be noted that I think my algorithm is slow. The proof in
the thesis is a reverse-engineered hack. :-) Finally, I don't think
we'll be in a position to have Axiom's language like B-natural
(described in ISSAC '94: "How to Make AXIOM into a Scratchpad" Jenks and
Trager). There will always be cases where users have to coerce (or
worse, cast e.g. Q -> Float). I don't think that my work is a panacea
for all ills but it should help and is mathematically sound in a large
number of day-to-day cases.

\start
Date: Thu, 18 Dec 2003 02:49:45 -0500
From: Tim Daly
To: list
Subject: audit

Gentlepersons,

I have completed a review of the axiom-changes.tar.gz file as posted
on your website (http://savannah.gnu.org/statement.html). All of the
changes appear to be valid. Therefore Axiom can be considered as
"clean" as of the time you did the diff.

\start
Date: Tue, 23 Dec 2003 15:57:09 +0100
From: David Mentre
To: list
Subject: Savannah is back

Hello Axiom users and developers,

I don't know if Axiom users ou developers have been annoyed but
Savannah, the infrastructure provided by the Free Software Foundation
(FSF) is now on-line after several weeks of down-time due to a system
crack.

We have now access to CVS trees and others development tools but several
parts are still missing, like the download area or the update of the Web
pages[1].

Tim Daly, Axiom project leader, has checked the Axiom sources and
confirmed that Axiom sources has NOT been compromised[2].

The Axiom project members will have to get used to new procedures and
will have to find new temporary solutions to lack of download area. So
expect a little more issues for the Axiom project is the short term.

Beside that, for the whole Axiom team, I wish you a nice christmas.

[1] http://savannah.nongnu.org/forum/forum.php?forum_id=2752
[2] http://mail.gnu.org/archive/html/axiom-developer/2003-12/msg00052.html

\start
Date: Tue, 23 Dec 2003 10:24:43 -0500
From: Tim Daly
To: David Mentre
Subject: re: Axiom is back

*,

I got email this morning from Brad Kuhn at FSF about Savannah.
He details new procedures for fetching and updating Axiom.
I am going to try these procedures (as well as a few project-lead
tasks I need to do) and then I'll send a note to these mailing
lists to clarify how you can again access Axiom.

The new procedures are mildly more painful but they are supposed to
prevent this kind of distruption in the future.

Happy holidays to all.

\start
Date: 23 Dec 2003 17:27:10 +0100
From: Wolfgang Zocher
To: list
Subject: Merry Christmas

A Merry Christmas to all developers, users and friends of Axiom from

-- 
Wolfgang Zocher  http://wzocher.bei.t-online.de/
Registered Linux User #337888 using Debian GNU/Linux

\start
Date: Sun, 28 Dec 2003 15:06:06 -0500
From: Tim Daly
To: Bill Page
Subject: documentation and the crystal 
Cc: Gilbert Baumslag

Bill,

per our long-lost prior discussion i've been documenting axiom.
the idea of documenting a system like axiom has several facets
as i've come to discover. what it means to document axiom has
a lot to do with why you are looking at the documentation. so
in some way the documentation has to be structured into something
like a large crystal egg with many facets. somehow we've got to
design a documentation structure that will allow the same basic
information to be used in dozens of different ways depending on
why you're looking at it. consider that axiom needs documenting:

of the internal structure of the system
  the data structures
  the logical layers (c, lisp, boot, spad)
  the functional separation (compiler, interpreter, browser)
  the system-dependent (sockets) vs independent (lisp)
of the external structure
  the interpreter
  the compiler
  the graphics
  the hyperdoc
  the openmath
  the documentation
of the algebra structure
  the packages
  the domains
  the categories
of the mathematical structure
  the subjects covered
  the theory underlying the subjects
  categories
of the computational mathematics structure
  intermediate expression swell
  simplification
  math types vs computational types
of the user structure (the book)
  the commands for naviation
  the commands for documentation
  the available math functions
of the programming language
  the compiler syntax
  the compiler semantics
  sequencing, conditionals, looping, file i/o
  domain construction
  categorical tests/constraints
of the testing structure
  the mathematics underlying the tests (CATS, computer algebra test suite)
  the actual tests
  boundary conditions
of the literature
  published algorithms
  published theory
  thesis work
of the program proofs
  underlying theorems and lemmas
  program proof books
  
i'm currently mucking around in the algebra structure. in particular
i've catted all of the algebra together into one file and am "reducing"
it to it's primitive structure. this alone is a daunting task as it
starts out with about a quarter million lines which i've slowly reduced
to about 100k lines so far. i'm doing a topological sort of the algebra
to uncover the actual type hierarchy with the idea that it can be
reduced to a lattice. as you recall this problem was done once before
in order to get axiom to compile from scratch.

getting down to this level of detail for documenting makes it clear
that current systems of documenting are hopelessly weak. somehow we
need to take advantage of the computer to leverage and reuse documentation
in creative ways. if we don't we'll just drown in endless documents. ibm
was famous for delivering shelves worth of documentation which was never
used. barnes and noble has whole bookcases of documents on linux. that
way lies madness.

in fact, documentation is probably the wrong idea. we need somehow to
be able to automatically generate information from some core that
represents the axiom system itself. 

so i'm thinking about a "crystal browser", that is, a browser where you
can gaze into a crystal that surrounds axiom. each facet represents a
generated view of the whole system. trivially we could have a facet
that shows algebra source code. we could also have a facet that shows
the type hierarchy, etc. so it is clear we can create automatic facets
with just the existing code and programs that do structure analysis.

more generally we could construct facets that walk into the pamphlets.
one facet could walk the biblio references, another could extract the
tex, a third could walk index terms to find all references to a single
term (e.g. ideal). particularly interesting would be facets that walk
the semantic structure of the system so you could pick out particular
kinds of ideals or proofs using ideals, etc. certain facets could be
used to impose order by certain metrics (like rainbows in real 
crystals). such rainbow facets could show the type lattice ordered
by layer (ala the structure in the src/algebra/makefile). yet more 
generally is that "literate programs" need to have sufficient structure 
to support the crystal.

in particular, we need to look at some technology that will do some
automated work for us. one that leaps to mind is a semantic network.
new code would be automatically classified into the system based on
several factors (some hand supplied, some derived from the code).

the idea that it is "just documentation" and "just a browser" is 
a weak notion but a good start. in general one would like to use
facets to CONSTRUCT new algebra, new booklets, new proofs, etc.
so both "documentation" and "browser" are the wrong words.

in 30 year computational mathematicians will need to be able to
deal with the complexity of all of the facets of documentation
i mentioned above. we need to construct tools which help organize
the system in ways that a mathematician can effectively use to do
research work.

the general "visual image" is of a large crystal which you can rotate.
every facet gives a different view of the whole of the axiom system.
thus, a "crystal" surrounding axiom.

hope your christmas went well.

\start
Date: Sun, 28 Dec 2003 15:23:58 -0500
From: Tim Daly
To: Bill Page
Subject: documentation and the crystal 
Cc: Gilbert Baumslag

Bill,

more top level speculation about the crystal leads to an interesting
problem. right now we have the image of books and standard page
navigation allows us to put "next", "previous", "index", etc on
each page as navigation buttons.

at the other extreme is the web where "forward" and "back" are all
of the possible navigation choices.

with a crystal one assumes that each facet is somehow "near" to the
adjacent facets. since this is a virtual object the facets don't need
to be all equally shaped (e.g. 6 sided). some facets could have many
possible neighbors (where information views are rich) and some could
have relatively few neighbors. the notion of "next" facet could be
specified by an angle measure of some sort. "forward" and "back"
become paths on the crystal surface. hyperlinking can move from any
facet to any other. in particular, viewing a "corner" could show
several facets at once (e.g. code, it's location in the lattice,
and the documentation for that code) which are all joined at a
virtual corner.

more generally one could embed one crystal within another. so the
inner crystal facet shows the type hierarchy and the outer crystal
facet looks at code for a particular item in the hierarchy. this 
is the 3-D notion of 1-D pipes, that is, information from one source
is filtered thru a pipe and shown thru another source. (e.g. grep
the algebra for "Category" and filter out only the lines that define
the domain).

the crystal is a virtual object which is really nothing more than
a way to organize hundreds of functions to manipulate code. the
crystal could be specified as a file containing its structure
definition where each facet is just a series of pipe-connected 
functions and a list of adjacent facets with navigation tags.
it could be built on top of a standard browser.

if done right the crystal should not be axiom specific, only the
functions need to know the details of their data sources.

\start
Date: Sun, 28 Dec 2003 22:06:41 -0500
From: Bill Page
To: Tim Daly
Subject: RE: documentation and the crystal 
Cc: Gilbert Baumslag

Tim,

Well, I have often used the time between christmas and
new years for thinking what I hope are radical and
forward looking thoughts. It sounds to me like you share
this tradition!

We had a pleasant christmas but work stress and "holiday
stress" combined to give Faye a serious case of pneumonia.
She has been confined to bed for the last three days (but
improving) so that has significantly dampened our enjoyment
of the extraordinarily mild weather ... I trust that in
your case things went a little better. :)

I have also been thinking about Axiom documentation. I
definitely agree that the book (or even the "IBM library")
metaphor is not adequate for a system as complex as Axiom.
I realize that you are probably writing in an online
"brainstorming" style, but have to admit that your "crystal"
metaphor also tends to leave me feeling a little cold.
However, when you said "semantic network" this connected!

There is a large and growing literature on the "semantic
web".

 http://www.w3.org/2001/sw/

"Definition: The Semantic Web is the representation of
data on the World Wide Web. It is a collaborative effort
led by W3C with participation from a large number of
researchers and industrial partners. It is based on the
Resource Description Framework (RDF), which integrates a
variety of applications using XML for syntax and URIs for
naming."

In fact, NAG (Mike Dewar) has been one of the leading
organizations promoting the concept of mathematics in
the context of the semantic web.

  =
http://monet.nag.co.uk/cocoon/openmath/meetings/eindhoven2003/index.html

See especially Mike's presentation on Monet
 
http://monet.nag.co.uk/cocoon/openmath/meetings/eindhoven2003/proceedings=
/de
war-monet.htm

----------

So rather than "crystal", I am inclined to think of
the Axiom documentation (and programs) as a "web". In
fact, the Axiom web could fundamentally reside on a
web application server such as

  http://www.zope.org

Zope provides a high level object-oriented environment
using tools that are not so different than Axiom itself
(Python).

  http://www.zope.org/WhatIsZope

Using these tools together with some of the standards
discussed by Mike Dewar, I could imagine configuring Axiom
(and in the longer term, the network of Axiom developers)
into a dynamic distributed active environment for
mathematics ... [Ok, that's as far as I go with the hype.]

But seriously, as strange as it might sound at first,
what more suits the multidimensional network structure
of Axiom itself better than the "web". And the further we
look into your "thirty year" time horizon, the more sense
this makes to me.


> Bill,
> 
> per our long-lost prior discussion i've been documenting 
> axiom. the idea of documenting a system like axiom has 
> several facets as i've come to discover. what it means to 
> document axiom has a lot to do with why you are looking at 
> the documentation. so in some way the documentation has to be 
> structured into something like a large crystal egg with many 
> facets. somehow we've got to design a documentation structure 
> that will allow the same basic information to be used in 
> dozens of different ways depending on why you're looking at 
> it. consider that axiom needs documenting:
> 
> of the internal structure of the system
>   the data structures
>   the logical layers (c, lisp, boot, spad)
>   the functional separation (compiler, interpreter, browser)
>   the system-dependent (sockets) vs independent (lisp)
> of the external structure
>   the interpreter
>   the compiler
>   the graphics
>   the hyperdoc
>   the openmath
>   the documentation
> of the algebra structure
>   the packages
>   the domains
>   the categories
> of the mathematical structure
>   the subjects covered
>   the theory underlying the subjects
>   categories
> of the computational mathematics structure
>   intermediate expression swell
>   simplification
>   math types vs computational types
> of the user structure (the book)
>   the commands for naviation
>   the commands for documentation
>   the available math functions
> of the programming language
>   the compiler syntax
>   the compiler semantics
>   sequencing, conditionals, looping, file i/o
>   domain construction
>   categorical tests/constraints
> of the testing structure
>   the mathematics underlying the tests (CATS, computer 
> algebra test suite)
>   the actual tests
>   boundary conditions
> of the literature
>   published algorithms
>   published theory
>   thesis work
> of the program proofs
>   underlying theorems and lemmas
>   program proof books
>   
> i'm currently mucking around in the algebra structure. in 
> particular i've catted all of the algebra together into one 
> file and am "reducing" it to it's primitive structure. this 
> alone is a daunting task as it starts out with about a 
> quarter million lines which i've slowly reduced to about 100k 
> lines so far. i'm doing a topological sort of the algebra to 
> uncover the actual type hierarchy with the idea that it can 
> be reduced to a lattice. as you recall this problem was done 
> once before in order to get axiom to compile from scratch.
> 
> getting down to this level of detail for documenting makes it 
> clear that current systems of documenting are hopelessly 
> weak. somehow we need to take advantage of the computer to 
> leverage and reuse documentation in creative ways. if we 
> don't we'll just drown in endless documents. ibm was famous 
> for delivering shelves worth of documentation which was never 
> used. barnes and noble has whole bookcases of documents on 
> linux. that way lies madness.
> 
> in fact, documentation is probably the wrong idea. we need 
> somehow to be able to automatically generate information from 
> some core that represents the axiom system itself. 
> 
> so i'm thinking about a "crystal browser", that is, a browser 
> where you can gaze into a crystal that surrounds axiom. each 
> facet represents a generated view of the whole system. 
> trivially we could have a facet that shows algebra source 
> code. we could also have a facet that shows the type 
> hierarchy, etc. so it is clear we can create automatic facets 
> with just the existing code and programs that do structure analysis.
> 
> more generally we could construct facets that walk into the 
> pamphlets. one facet could walk the biblio references, 
> another could extract the tex, a third could walk index terms 
> to find all references to a single term (e.g. ideal). 
> particularly interesting would be facets that walk the 
> semantic structure of the system so you could pick out 
> particular kinds of ideals or proofs using ideals, etc. 
> certain facets could be used to impose order by certain 
> metrics (like rainbows in real 
> crystals). such rainbow facets could show the type lattice 
> ordered by layer (ala the structure in the 
> src/algebra/makefile). yet more 
> generally is that "literate programs" need to have sufficient 
> structure 
> to support the crystal.
> 
> in particular, we need to look at some technology that will 
> do some automated work for us. one that leaps to mind is a 
> semantic network. new code would be automatically classified 
> into the system based on several factors (some hand supplied, 
> some derived from the code).
> 
> the idea that it is "just documentation" and "just a browser" is 
> a weak notion but a good start. in general one would like to 
> use facets to CONSTRUCT new algebra, new booklets, new 
> proofs, etc. so both "documentation" and "browser" are the 
> wrong words.
> 
> in 30 year computational mathematicians will need to be able 
> to deal with the complexity of all of the facets of 
> documentation i mentioned above. we need to construct tools 
> which help organize the system in ways that a mathematician 
> can effectively use to do research work.
> 
> the general "visual image" is of a large crystal which you 
> can rotate. every facet gives a different view of the whole 
> of the axiom system. thus, a "crystal" surrounding axiom.
> 
> hope your christmas went well.

\start
Date: Mon, 29 Dec 2003 15:14:36 +0100
From: David Mentre
To: Tim Daly
Subject: Re: [Axiom-math] documentation and the crystal
Cc: Gilbert Baumslag

Hello Tim and Bill,

You are right, holidays are always a good period to dig into new ideas.

I think I have the same approach as both of you, Tim and Bill. Regarding
the Semantic Web, Bill, I think it is a interesting technology that we
might look at and use as a reference. But for me, it is just a
technology. We need to refine first our own ideas on what we want to not
lose our 30 years goal. That's said, you are right that smart people
have invested time in this technology and it would be stupid of us to
not resuse the wheel.

Tim, regarding your crystal approach, I particularly like it, at least
for the several "facets" to look through one complex system. However, I
would prefer to use a graph-like approach, which in my opinion is
probably more related to the Knowledge Network you have spoken about.

Usually, people have a top-down approach: they modelize a system
abstractly (using more or less formal notations as UML or SDL) and then
refine them through the actual code. But, in the case of Axiom, we need
the reverse. We need to start from concrete objects (files, lines of
source code) and add semantics to climb levels of abstraction. Of
course, you follow different ladders, in the sense that understanding
the compiler or the algebra would need different information and is
structured differently, thus the different crystal facets of Tim.


More concretely, I would propose the following approach:

 1. start from parsers for the src/ directory. Parse directory structure
    and each file, categorize them (boot, lisp, spad, ...) and construct
    basic abstractions (list of lisp and boot functions; list of spad
    categories, domains and operators; ...)

 2. from information extracted in step 1, construct one or several
    representation (knowledge graph for example) with the found
    semantics (name and body of a function for example), probably using
    a standard technology as W3C semantic web

 3. the "Axiom explorer" (Tim, Bill or I) is interested in a specific
    "crystal facet" (e.g. the compiler). He builds another tool
    (e.g. call-graph analyzer for lisp code) which in turn is used to
    construct a new knowledge graph, concretized in additional
    information in the W3C semantic web.

 4. Within the abstraction level build in step 4 (or 2), the "Axiom
    explorer" adds its own knowldege (e.g. this set of functions is used
    to parse Spad code, this other set is used for type analysis, ...)
    to the semantic web.

 5. using previous abstraction levels and probably building a new tool,
    the "Axiom explorer" iterates, climbing abstraction levels, until he
    reaches his own goal.

Repeat steps 1 to 5 with enough people to cover Axiom from A (Algebra)
to Z (zerodim). 

Ok. I might be a bit optimistic and what I have said might appear more
than abstract, but this is the current state of might thoughts. :) 

Even if you do not like above ideas, I think following "principles" are
needed for a documentation system for Axiom (principles already
formulated by Tim is his first email):

 o separation principle: in engineering, people separate a complex issue
   in _independant_ sub-issues to be able to understand them and solve
   them independently. We probably need to deconstruct Axiom in
   independent (or a least loosely connected) sub-systems to be able to
   understand them (i.e. the different crystal facets of Tim);

 o "build on giant shoulders" principle: we need a way to reuse
   knowledge from other "crystal facets". For example, I would use Tim
   knowledge of the internals of Axiom to understand how the compiler
   compiles a given portion of the algebra;

 o automation principle: Axiom is too big to add information manually on
   each function, each object, etc. We need tools to annotate a set of
   objects given a selection criteria (e.g. all operators in this Spad
   domain). 

New year wish: I'll try to write and "show you the code" for above
ideas. :) My own todo for this subproject of Axiom would be:

 - learn more about W3C Semantic Web (thank you Bill for the pointer)

 - find or write tools to manipulate Semantic Web (it might be Emacs
   with a proper mode or a more elaborated graphical tool)

 - apply above approach, starting from directory structure in src/
   directory of Axiom.

 - from this first experiment, think about what would need to be
   "standardized", like common dictionnary or vocabulary, etc. Beyond
   usual technological issues, I think this point is one fo the harder
   point. How to build a set of knowledge that will be still useful in
   30 years from now?


By the way, does anybody knows about a library to *draw* and manipulate
arbitary graphs (in Common Lisp, C++, ML, ...) in a user interface? I
know about DOT or VCG but they dot not match my needs: I would like to
draw a graph, know when the user click on a graph node or edge and react
accordingly. Does anybody know where I could find such an already made
tool? Any knowledge of a browser for the W3C semantic web?

\start
Date: 29 Dec 2003 16:07:32 -0500
From: Camm Maguire
To: Tim Daly
Subject: si::sgc-on t in Axiom?

Greetings!  I'm just wondering if anyone has had experience building
axiom with the 'sgc' feature in gcl turned on.  This is a write memory
barrier intended to speed up garbage collection.  My initial attempts
are leading axiom (when compiling OUT.NRLIB) to complain with

$ ../.././obj/linux/bin/interpsys
GCL (GNU Common Lisp)  (2.6.1) Thu Nov  6 16:23:13 UTC 2003
Licensed under GNU Library General Public License
Dedicated to the memory of W. Schelter

Use (help) to get some basic information on how to use GCL.

(AXIOM Sockets) The AXIOM server number is undefined.
-----------------------------------------------------------------------------
   Issue )copyright to view copyright notices.
   Issue )summary for a summary of useful system commands.
   Issue )quit to leave AXIOM and return to shell.
Monday December 29, 2003 at 19:53:49 
-----------------------------------------------------------------------------
 
   Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/compress.daase..   Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/interp.daase..
   Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/operation.daase..
   Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/category.daase..
   Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/browse.daase..
(1) -> )co OUT.spad
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/apply.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/c-doc.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/c-util.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/profile.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/category.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/compiler.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/define.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/functor.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/info.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/iterator.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/modemap.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/nruncomp.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/package.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/htcheck.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/xruncomp.
   Compiling AXIOM source code from file 
      /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/OUT.spad using old 
      system compiler.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/parsing.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/bootlex.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/def.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/fnewmeta.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/metalex.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/metameta.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/parse.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/postpar.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/postprop.
   Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/preparse.
   OUT abbreviates package OutputPackage 
------------------------------------------------------------------------
   initializing NRLIB OUT for OutputPackage 
   compiling into NRLIB OUT 
   processing macro definition E ==> OutputForm 
   processing macro definition putout ==> elt(Lisp,mathprint) 
   compiling exported output : OutputForm -> Void
Time: 0.01 SEC.

   compiling exported output : String -> Void
Time: 0 SEC.

   compiling exported output : (String,OutputForm) -> Void
Time: 0.03 SEC.

   compiling exported outputList : List Any -> Void
   Internal Error
   Error while instantiating type Any 

protected-symbol-warn called with (NIL)


I obviously haven't done much research yet -- just asking if anyone
else has experience.

\start
Date: 29 Dec 2003 17:04:13 -0500
From: Camm Maguire
To: Tim Daly
Subject: Re: si::sgc-on t in Axiom?

Greetings!

Please disregard this message -- it appears to be working with the
latest SGC fix I've committed for some UT people.  This fix is already
in 2.6.1-19 which is about to (in a few days) become the official
2.6.2 release. I'll post comparisons if/when I get a chance.

Take care,

Camm Maguire writes:

> Greetings!  I'm just wondering if anyone has had experience building
> axiom with the 'sgc' feature in gcl turned on.  This is a write memory
> barrier intended to speed up garbage collection.  My initial attempts
> are leading axiom (when compiling OUT.NRLIB) to complain with
> 
> $ ../.././obj/linux/bin/interpsys
> GCL (GNU Common Lisp)  (2.6.1) Thu Nov  6 16:23:13 UTC 2003
> Licensed under GNU Library General Public License
> Dedicated to the memory of W. Schelter
> 
> Use (help) to get some basic information on how to use GCL.
> 
> (AXIOM Sockets) The AXIOM server number is undefined.
> -----------------------------------------------------------------------------
>    Issue )copyright to view copyright notices.
>    Issue )summary for a summary of useful system commands.
>    Issue )quit to leave AXIOM and return to shell.
> Monday December 29, 2003 at 19:53:49 
> -----------------------------------------------------------------------------
>  
>    Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/compress.daase..   Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/interp.daase..
>    Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/operation.daase..
>    Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/category.daase..
>    Using local database /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/../../src/share/algebra/browse.daase..
> (1) -> )co OUT.spad
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/apply.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/c-doc.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/c-util.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/profile.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/category.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/compiler.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/define.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/functor.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/info.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/iterator.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/modemap.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/nruncomp.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/package.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/htcheck.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/xruncomp.
>    Compiling AXIOM source code from file 
>       /fix/g/camm/axiom/axiom-0.0.0cvs/int/algebra/OUT.spad using old 
>       system compiler.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/parsing.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/bootlex.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/def.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/fnewmeta.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/metalex.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/metameta.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/parse.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/postpar.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/postprop.
>    Loading /fix/g/camm/axiom/axiom-0.0.0cvs/mnt/linux/autoload/preparse.
>    OUT abbreviates package OutputPackage 
> ------------------------------------------------------------------------
>    initializing NRLIB OUT for OutputPackage 
>    compiling into NRLIB OUT 
>    processing macro definition E ==> OutputForm 
>    processing macro definition putout ==> elt(Lisp,mathprint) 
>    compiling exported output : OutputForm -> Void
> Time: 0.01 SEC.
> 
>    compiling exported output : String -> Void
> Time: 0 SEC.
> 
>    compiling exported output : (String,OutputForm) -> Void
> Time: 0.03 SEC.
> 
>    compiling exported outputList : List Any -> Void
>    Internal Error
>    Error while instantiating type Any 
> 
> protected-symbol-warn called with (NIL)
> 
> 
> I obviously haven't done much research yet -- just asking if anyone
> else has experience.

\start
Date: Tue, 30 Dec 2003 11:24:50 -0500
From: Tim Daly
To: Bill Page, Alexei V. Otboyev, Gilbert Baumslag, David Mentre
Subject: crystal and the semantic web

*,

Sorry for the delay in reply. My inbound mail queue got silently hosed.
Working on it now.

\start
Date: Tue, 30 Dec 2003 10:38:48 -0500
From: Bill Page
To: Tim Daly
Subject: RE: crystal and the semantic web
Cc: Gilbert Baumslag, Alexei V. Otboyev

On Tuesday, December 30, 2003 11:25 AM Tim wrote:
> 
> Sorry for the delay in reply. My inbound mail queue got 
> silently hosed. Working on it now.
> 

I guess that it is the season to get "hosed", silently,
in the night ... <grin>

\start
Date: Tue, 30 Dec 2003 19:07:15 +0100 (CET)
From: Bertfried Fauser
To: Tim Daly
Subject: Re: documentation and the crystal 
Cc: Gilbert Baumslag, Bill Page

Dear Christmas holiday workers!

beside wishing all the best for the new year, I would like to ask a few
more questions about axiom documentation.

a) To me its not even clear how to structurize mathematics. One cxan
build mathematics on sets, hence the Bourbaki approch, or even better (in
my eyes, but equivalent in formal strength) on the *function first*
principle. What to choose for axiom? (In fact the set approch is build-in)

b) I do not see how you can automatically assign semantics to data
strutures etc. I think, one has at least to have a sort of *semantic
typing* during the documentation of axiom (code). Hence every piece of
documentation should come with a semantic type (or multiple such types)
which finally allow to put a direction into the crystal looking glass.

c) Algorithms should be plain and readable and not only be available in
code form (if even this way)

d) To me it would be much more natural to look at the documentation like a
big database and ask SQL like questions. EG:

> select from AXIOM algorithm where domain has commutative;

or such. And then get a sort of (semantic? web?) document which allows to
go deeper into an algorithm, eg, thet should be lionks to al faces of the
crystal which make sense to look at.

e) The system will not be more smart than its designers / users. I do not
see how an automated method will derive anything beyong mere syntactical
sorting. To be frak I have no idea how to reach the above needs.

f) As a probably managable project, it would be of utmost importance for
me (and other mathematically interested, who are stupid programers) to
have just the functionality described above for teh algebra lattice. I had
such an email with David Mentre, where some of the needs were discussed.

g) I do not think that *graph* is the best thing to have. One would need a
sort of *matroid*, ie. a mathematical sound structure which keeps trak of
all sorts of *dependencies* (assuming that independent objects/data are
unrelated) Matroids allow you to keep trak of a minimal set of relations
between objects etc. (You may think of a combinatorial geometry, where the
points/lines/... may or may not be related.
	It might be foolish just to have nods and edges, but one hast to
have an n-dimesnional structure of nodes, edges, faces, volumes, ... where
the semantic meaning could be attached to the *dimensionality* of the
object. This would allow to trace up and down iinto the complexity of teh
system if eg. code has teh smallest dimensionality (say 1 dim), algorithms
a hiher (say 3-dim (to let space for future enhancements)) and
documentation of algorithms or a proof it works etc even higher such
dimenionality. A *browser* would offer to display or hide the complexity
away from a user or show it to him.
	Say a novice user will be confused to see all details but needs
quite practical help and may be even examples (like in the book)

Sorry for not beeing as structures as you were, but I had no time to think
over Christmas ;-))

\start
Date: Tue, 30 Dec 2003 11:40:51 -0500
From: Tim Daly
To: Bill Page
Subject: crystal and the semantic web
Cc: Gilbert Baumslag

Bill,

Sorry to hear about Faye's illness. I've been down for a week with
the flu and then useless for nearly a week with a bad head cold of
which I still have an annoying cough. It's a bad season for it.

I worked on a combination rule-based program/knowledge-representation
program called KROPS (a combination of KREP and OPS5) years ago. It
had a lot of really nice features including automatic knowledge
classification. KROPS is an instance of a semantic network, thus my
interest in the subject. It has the unique property that rules and
knowledge chunks were dual representations. That is, you could write
in either style and update the underlying graph data structure.

I'll look at the semantic web info you sent.

\start
Date: Tue, 30 Dec 2003 17:58:05 +0100
From: David Mentre
To: Tim Daly
Subject: Re: crystal and the semantic web
Cc: Gilbert Baumslag, Bill Page

Hello Tim,

Tim Daly writes:

> I worked on a combination rule-based program/knowledge-representation
> program called KROPS (a combination of KREP and OPS5) years ago. It
> had a lot of really nice features including automatic knowledge
> classification. KROPS is an instance of a semantic network, thus my
> interest in the subject. It has the unique property that rules and
> knowledge chunks were dual representations. That is, you could write
> in either style and update the underlying graph data structure.

Have you a reference to a concise article descibing KROPS? Is this
software available as free software?

> I'll look at the semantic web info you sent.

I've started with : RDF Primer: http://www.w3.org/TR/rdf-primer/

\start
Date: Tue, 30 Dec 2003 18:33:14 +0100
From: David Mentre
To: Tim Daly
Subject: Re: crystal and the semantic web

David Mentre writes:

>> I'll look at the semantic web info you sent.
>
> I've started with : RDF Primer: http://www.w3.org/TR/rdf-primer/

And this sub-link of previous Bill's link is also useful:
 http://www.w3.org/2001/sw/Activity

\start
Date: 30 Dec 2003 17:27:46 -0500
From: Camm Maguire
To: Tim Daly
Subject: Speeding up compilation of EXPEXPAN.spad

Greetings!  I've looked at this a bit with si::*notify-gbc* set to t,
and am noticing an enormous amount of (apparently bignum) garbage
generation.  (si::set-gmp-allocate-relocatable t) helps, but I'm
wondering where this is coming from, and whether a well placed
compiler optimization might eliminate it.  Can someone point out the
time critical code snippet in lisp?

\start
Date: Wed, 31 Dec 2003 00:35:16 -0500
From: Tim Daly
To: Bill Page
Subject: crystal and the semantic web
Cc: Gilbert Baumslag

Bill,

I realize that I'll have to "sell" the crytal idea as it is new
and unknown. I've been trying to organize Axiom in various ways
to make it more useful in the long term and this idea "crystalized"
so I'm going to run with it for a while. Feel free to ignore me or
try to persuade me otherwise.

More shower committee discussion leads to the following, somewhat
more "grounded" thoughts:

COLLECT COMPILER INFORMATION

Suppose we build a library of tools whose only purpose is to present
a slice of information. 

In particular, suppose we "instrument" the compiler so that it will
give us data about a given domain like:

the domain name
the domain signature
the list of exports
the list of "required" domains
etc

then we can use the compiler to automatically build databases about
the algebra. Axiom already does this with spad code (in the NRLIB
dirs). We can instrument the boot compiler in the same way.
We can even instrument the lisp compiler to dump information.
Once these databases exist and are automatically maintained we can
automatically build cross-reference and index tables. This information
can easily be gotten by using the "asq" command line program which
currently exists and queries the databases. Most of this machinery
already exists at the spad level but needs to be built at the boot
and lisp levels.

A simple crystal facet would look at each of these lists (eg the list
of domain signatures). Related simple facets could (a) select a single
domain to inspect or (b) select domains by regular expression on the
signatures or (c) show the hierarchy of domains using this domain or
(d) show the hierarchy of domains used by this domain or (e) show the
exports, etc. 

Notice that all of this information is available from the compiler
databases and can be extracted by an asq command line program.
so we get 
  src -> compiler -> databases -> asq -> specific facet display
and since this process is driven from the facet we can merge
the crystal front end with a makefile middle and a database back end
so we get:
 special information  -> makefile -> specific facet display

So where does that get us? Well, now we have a pretty little browser
that can walk all over the algebra. Useful but not very novel. The
key flaw is that it doesn't connect to anything except machine-readable
code. It does have the feature that we can write makefile-style 
connectors between the database support code (asq) and the facets.
In fact, what defines a facet is two pieces of information:
  the data that an asq command can extract
  the relationship of commands required to construct facet data

LATEX STRUCTURE INFORMATION

Clearly we want more than just a fancy code-browser. Another need
is to connect the latex documentation to the code. Currently we
walk over a pamphlet document with noweb to create a stream which
can extract code from the document. And we have a higher level 
function (booklet) that can assemble the pamphlet before extracting
information.

We can easily create a "notangle" facet to show the code, a
"noweave" facet to show the latex (possibly in an editor like texmacs),
and a dvi viewer to see the tex output facet. We can even construct
a "booklet" facet that connects parts or whole pamphlets.

But this leaves several huge and important gaps. Pamphlet files have
several semantically different sections. We need to recognize, parse,
and treat these sections differently.

For example, in src/algebra/dhmatrix.spad.pamphlet we describe the
Axiom code with long sections of mathematics. We would like to 
connect the mathematics with the axiom algebra in more than an 
incidental way. Having them in the same pamphlet file makes at
least a marginal connection but we need to do more.

If we look at the technologies available we could use several.
The easiest technique is to use latex tags such as \ref and \index
to mark terms that can be added to a database during search. More
generally we could invent tags that give us some connection between
the english text and the embedded algebra.

The next technique is to try to "read" the actual text by machine.
There are also natural language parsing technologies (e.g. a chart
parser) that will give us some minimal information. I don't think
the technology is good enough to be useful. I have used this idea
in the past to read english text, form a "concept" of the sentence
and automatically classify the concept in a semantic network. This
would be easier for mathematical sentences rather than general
english as you could pre-populate the semantic network with appropriate
concepts from mathematics.

The next technique is to require additional tags in the document.
This approach amounts to decorating the latex document with
type information. For example, instead of:
$x+y$
we could add the axiom types:
$\type{x}{UnivariatePolynomial}
 \type{+}{UnivariatePolynomial}
 \type{y}{UnivariatePolynomial}
$
which we could use to connect back to Axiom. This assumes that we
know the Axiom types which might not be true for most mathematical
expressions encountered in text. Perhaps this might be a useful
place to consider embedding OpenMath expressions.

Another technique is to try to constrain the written expressions.  We
can require that Axiom be able to parse expressions in pamphlets. That
way we can use the compiler to tell us what types are in the text and
how they connect to Axiom. This lends itself to automation with the
current tools. It also constrains the kinds of expressions you can use
in documentation to well-formed expressions.  A fair number of math
expressions in books are not well-formed.

The struggle here is to figure out a way to move from the latex math
to the axiom math and back again. Literate programs are a weak but
widely available technique.

At a different semantic level, that of the whole document, we can use
bibliographic references to connect the documents. This could range 
from hardly more than hyperlinks to more general semantic network links.

SPECIAL INFORMATION STRUCTURE

I'm advocating building an information source (database is really a
poor choice of words). It should be searchable using a semantic query
(parse and form a concept, "classifying" the concept and returning
nearest-neighbor concepts), using hierarchical structure (standard
hierarchical databases) and by keyword/grep/hashtable kind of
walks. Graph-relationship walks (like nearest-neighbor) should also be
possible.  Standard relational queries are the least interesting way
to search this kind of information. I see this as a highly linked
graph object (a ball of string) with pointer tables to support other
views.

GENERAL MATHEMATICS STRUCTURE

Moving away from the specifics of a particular latex document we can
ask about facets which are "structural". We'd like to be able to see
the overall structure of parts of the world, usually in some sort of
a lattice.

Lattice-style facets can be constructed (by machine) that detail the
inheritance hierarchy in Axiom and show the relationship of types and
categories.  Lattice-style facets can also be constructed (by hand)
that show the relationship of the mathematics covered by the system.
Links could be constructed (again, by hand) that relate the real
concept (e.g. Ring) to the Axiom concept (e.g RING). If these were done
as semantic network "concepts" you could walk about the system using
the lattice. The various lattices could form the skelton of a concept
framework in a semantic network (our ball of string above).

Indeed, the CATS (Computer Algebra Test Suite) eventually must extract
some sort of taxonomic organization of the mathematics used in the
test suites. This would be very similar to the NIST taxonomy for
numeric suites and would provide a concrete connection between the
mathematics and the available system functions.

PERSONAL FACETS

Eventually all of this machinery is justified by the need to organize
a very large (I'm assuming a 100x expansion of math in axiom) pile of
information so it can be used by one or a group of mathematicians to
do research. 

It needs to be able to accept dynamic changes (assume hundreds of
papers are published as literate programs every year and become 
available at the rate of one per day) without human intervention.
I'm assuming that literate programs get published and become 
available to global computational math research pipeline which can 
be reviewed and searched.

Crystal needs to be able to have several "research" threads available
at any given time. That is, I need to be able to highlight, remember,
collate, and organize existing work in multiple areas at the same
time. I want to be able to have several "papers" in progress and I
want Axiom to remember that I've visited certain areas of math
(e.g. crytography using group theory). I want the system to strip
out this part of math and Axiom and present it to me on one or 
several facets so I can work on it. Eventually I want MY Axiom to
"know" (in some mechanical sense) what kinds of math I find interesting,
to search for literate papers related to that kind of mathematics, and
to extend the system to cover published results by selecting and embedding
literate programs into my copy of Axiom. 

The more I work with my copy of Axiom the more specific it should become
for me. The internal semantic network should eventually develop very
clear, specific meanings (concept clusters) for terms and ideas that
are important to me. Facets should be able to be dynamically created
based around these dynamically developed concept clusters.

(As an aside eventually this clustering of concepts makes it difficult
for two Axiom systems to communicate. Consider an example of an electrical
engineer and a computer programmer. Both use the concept of a "register".
Eventually the EE's system has a concept cluster around the term "register"
that includes ideas of NAND gates, transistors, CMOS, ECL, etc. Eventually
the CS's system has a concept cluster around the term "register" that
includes ideas of register windows, base registers, stacks, etc. Notice
that the same "word" has completely different meanings to the two systems.
They will eventually become so specific that the concept nets cannot be
merged. Perhaps this is why it is so hard for geeks to talk to women :-) )

Well, I've wandered off into the weeds and it's time to work on something
useful.

\start
Date: Wed, 31 Dec 2003 00:36:41 -0500
From: Tim Daly
To: Camm Maguire
Subject: Re: Speeding up compilation of EXPEXPAN.spad

You're seeing bignum computation during the compile of EXPEXPAN?
What a wildly curious observation. I can't imagine why bignums
are even involved in compiling. I'll look into it.

\start
Date: Wed, 31 Dec 2003 00:53:39 -0500
From: Tim Daly
To: David Mentre
Subject: Re: crystal and the semantic web
Cc: Gilbert Baumslag, Bill Page

David,

KROPS was an effort I did while at IBM. I worked on two separate 
research areas, expert systems which involved the OPS5 rule based
programming using a Rete. This eventually became a product (that
sold 2 copies) called Eclipse. The second effort was in Knowledge
Representation involving KREP, a research effort led by Eric Mays.
This eventually became a spinoff company.

I found a unification of the two ideas, a Rete pattern-match network
and a semantic network. I created a language called KROPS (KREP-OPS5)
which had the interesting property that rules and concepts were dual
representations of the same object. So concepts could be classified and
expressed as rules and rules could be entered and classified as concepts.
IBM used KROPS as the basis for the FAME system (Financial and Marketing
Expert). FAME died and KROPS went with it. 

I have no idea what became of the software.

The only published report on KROPS that I helped author was:

"Integrating Rules and Inheritance Networks in a Knowledge-Based Financial
and Marketing Consultation system", T. Daly, J. Kastner, E. Mays, in
Proceedings Hawaii International Conference on System Sciences, HICSS-21,
Hawaii, Jan 1988

\start
Date: Wed, 31 Dec 2003 02:53:05 -0500
From: Tim Daly
To: Bertfried Fauser
Subject: Re: documentation and the crystal
Cc: Gilbert Baumslag, Bill Page

>a) To me its not even clear how to structurize mathematics. One cxan
>build mathematics on sets, hence the Bourbaki approch, or even better (in
>my eyes, but equivalent in formal strength) on the *function first*
>principle. What to choose for axiom? (In fact the set approch is build-in)

Well, I am assuming that we're only trying to structure "computational"
mathematics. That is, we're only trying to figure out how to organize
the pieces of mathematics we can compute. The NIST organization in the
U.S. government did a similar organization of information for numerical
mathematics years ago. Thus you can now find a specific index to classify
a numeric routine that will do Runga-Kutta integration on a certain class
of functions. A second index will find routines that do Simpson integration.

Since we've limited the mathematics to computational forms we should be
able to collect algorithms and classify them just as NIST has already done.
Thus we would find different classifications for Clifford-algebra algorithms
vs Hopf-algebra algorithms.

>b) I do not see how you can automatically assign semantics to data
>strutures etc. I think, one has at least to have a sort of *semantic
>typing* during the documentation of axiom (code). Hence every piece of
>documentation should come with a semantic type (or multiple such types)
>which finally allow to put a direction into the crystal looking glass.

Indeed assigning semantics is hard. Three approaches leap to mind.
The first is by keyword assignment in pamphlet files. This is weak but
easy (although time consuming). The second is to use the compiler to
try to parse the mathematical expressions in the tex file and assign
meaning (types) to the symbols.  This is hard but might be helped
along if we use OpenMath representations of the mathematics. I'm not
sure if OpenMath is strong enough to handle general mathematical
expressions. The third is to use a chart-parser and semantic network
software to try to read and classify the mathematics.

I did an effort similar to the third case while at IBM. We built a 
system called "Susan" which read english-language email, parsed it
using a chart-parser, constructed a semantic "concept", classified it,
and used the nearest-neighbor concepts to help direct further parsing.
Eventually the whole email and it's paragraphs, sentences, and phrases
became concepts. Once the email was classified (e.g. does it set up a
meeting? does it require an answer? does it have a deadline? is it from
someone important?) the email was assigned to a "basket". So email with
a "deadline" went into a "tickle file" that would begin reminding you
of the deadline several days in advance.

We could build a Susan-like system that could handle some portion of
the mathematics because we have the advantage of working in a limited
domain. Since we know the domain we can prepopulate the semantic 
network with concepts. These concepts can then be used to direct the
chart-parser toward a correct parse of a sentence. This would be great
fun but falls under the "real research" category and thus will never
get funded :-)

>c) Algorithms should be plain and readable and not only be available in
>code form (if even this way)

I absolutely agree. Indeed I would hope to see several versions of
explanation for algorithms. My current "best practices" example is
"Primes is in P" by Agarwal, et. al.
(http://www.cse.iitk.ac.in/news/primality.pdf) They have the closest
thing to a literate program that embodies the kind of information I
hope to see. They show the theory, the pseudocode, the bounding
conditions, the proof, a complexity analysis and references. I only
wish they had reduced the pseudocode to real code and published the
original tex. Other than that I feel they've "set the standard" for a
good computational mathematics paper.

>d) To me it would be much more natural to look at the documentation like a
>big database and ask SQL like questions. EG:
>
>> select from AXIOM algorithm where domain has commutative;
>
>or such. And then get a sort of (semantic? web?) document which allows to
>go deeper into an algorithm, eg, thet should be lionks to al faces of the
>crystal which make sense to look at.

I agree that a large database will eventually underlie the whole of the
system. I'm not sure that a relational database is an entirely useful
model. Yes, it is complete but it is a very awkward way to structure
mathematical questions. A semantic network concept would allow you to
"fill in" as much as you know about something (e.g. domain, commutative),
"classify it", and then find all of the concepts that it dominates.
Some of these concepts would be the domains you want.

>e) The system will not be more smart than its designers / users. I do not
>see how an automated method will derive anything beyong mere syntactical
>sorting. To be frak I have no idea how to reach the above needs.

We're not trying to make it smarter we're trying to make it useful.
At the moment Axiom is just a huge pile that is nearly inert. It
includes primitive query functions but little else. If you are going
to use a large system (scale Axiom by 100x) you need to be able to
move around it in much more fluid ways.  As a researcher you sometimes
wander into a library and browse thru books that might be related to
some half-formed idea. Crystal is an attempt to bring "wandering" into
the computer age.

Crystal should do at least 3 things. First, it should richly classify
and cross-connect the various sources of information (source code, an
algorithm, proofs of that algorithm, a complexity analysis, pointers
to related algorithms, the domain of application of the algorithm,
required inputs and their types, explanations of the theory behind the
algorithm, published results using the algorithm, domains which use
the algorithm, boundary test cases, examples of its use, etc. each
of which could be a kind of facet).

Second, Crystal should watch how you wander and work to taylor it's
answers to your interests. If I am looking at information on
cryptopgraphy and group theory I don't want to know anything about the
weakly related concepts (e.g. the probabilities in threat
matricies). I would like the system to remember what I've looked at
(so I can find it again), to remember how I got there (so I can start
from some point and go off in a different path), and to "suggest"
(nearest neighbor) things that are related but I didn't know (or
forgot) to look for. And I'd like a facet automatically created which
is related to the interest and possibly other facets related to the
details. After all, if I'm trying to write a paper I'm usually piling
up a bunch of related things I need as background. Computers are good
at keeping track of things and we need to organize the tracks.

Third, it should be capable of finding things in the stream of
computational mathematics literature that I would find "interesting".
I'm assuming that computational mathematics will eventually be online
and organized in some computer reachable way. Hopefully as literate
programs. So Crystal ought to be able to constantly be looking thru
all of the recently published papers for anything that fits my idea of
interesting.  Trivially this involves a keyword search of papers that
get published but could be much more complex. In this way I can build
up a local library of references that make my Crystal searches more
interesting. And since they are literate programs I'm also building
up the mathematics for my areas of interest.

We're capable of doing portions of the first idea with the current
tools but it requires people who write code to make an effort to
make their work more accessible to the machine. We don't even have
a standard outline of what should be in a literate program paper yet.
We need to build a few papers, build some technology to exploit the
papers, rebuild the papers and the technology, etc. until we get to
the point where we find the system useful.

>f) As a probably managable project, it would be of utmost importance for
>me (and other mathematically interested, who are stupid programers) to
>have just the functionality described above for teh algebra lattice. I had
>such an email with David Mentre, where some of the needs were discussed.

In the very short term (6 months?) we're likely to have the algebra
lattice available. David has discussed various ways of displaying such
a beast (it has 1100+ nodes at the top level). I'm looking at ways of
building a computational math classification based on test cases from
several different computer algebra systems (CATS). The combination of
the machine-built algebra lattice and the hand-built classification
system should give us a simple prototype to play with. We need a
simple browser tool. I looked at leo which is an open source literate
browser but haven't decided how to leverage it yet. We need a backend
(database/semantic network). If I can remember the insight from KROPS
perhaps I can reproduce it.

>g) I do not think that *graph* is the best thing to have. One would need a
>sort of *matroid*, ie. a mathematical sound structure which keeps trak of
>all sorts of *dependencies* (assuming that independent objects/data are
>unrelated) Matroids allow you to keep trak of a minimal set of relations
>between objects etc. (You may think of a combinatorial geometry, where the
>points/lines/... may or may not be related.
>	It might be foolish just to have nods and edges, but one hast to
>have an n-dimesnional structure of nodes, edges, faces, volumes, ... where
>the semantic meaning could be attached to the *dimensionality* of the
>object. This would allow to trace up and down iinto the complexity of teh
>system if eg. code has teh smallest dimensionality (say 1 dim), algorithms
>a hiher (say 3-dim (to let space for future enhancements)) and
>documentation of algorithms or a proof it works etc even higher such
>dimenionality. A *browser* would offer to display or hide the complexity
>away from a user or show it to him.

I like the idea of dimensionality (faces, volumes). I don't believe I've
ever seen such an idea applied to a semantic network before. In a semantic
network "concepts" (data structures) tend to be very near each other
(counting links) if they are "similar". Semantic networks tend to have
"clusters" of concepts that express a particular "idea". These clusters
form because an idea has many similar ways of being expressed so there
are many links. Different "ideas" will have relatively fewer links 
between them. The closest connection to dimensionality is probably that
a "volume" can be a "open ball" formed around a cluster of concepts.
If you measure the number of links that cross the boundary of the 
volume and minimize that number you can count the number of "concepts"
inside the "volume". Thus you can have "big ideas" (a really rich
cluster of concepts) and "small ideas" (a small cluster).

I'm unfamiliar with the matriod idea. What is it? An n-dimensional 
spreadsheet? 

>	Say a novice user will be confused to see all details but needs
>quite practical help and may be even examples (like in the book)

I expect novice users to spend most of their time in the examples facets
so we should probably be sure they work :-)

>Sorry for not beeing as structures as you were, but I had no time to think
>over Christmas ;-))

Every so often I try to look toward the 30 year horizon to decide if we're
building for the future. I was too sick to type over christmas so I had a
lot of time on my hands to think. Now that I'm back to typing I'm trying
to "ground" the ideas in working code.

Shortly we'll all have access to several terabytes of disk, a terabyte of
storage, and a terahertz of cpu cycles connected to gigabyte bandwidth.
You'll be able to put virtually every paper ever published on a local
disk drive and search them in seconds. Imagine what will be possible
in 30 years. Libraries are doomed, books will be electronic. Browsing
will be incredibly painful if we don't get creative.

\start
Date: Wed, 31 Dec 2003 03:00:38 -0500
From: Tim Daly
To: Alexei V. Otboyev
Subject: numeric cosine solution

>Hi!
>
>How can I solve some equation numericaly? For example, I need to find solution of cos(x)=x and I know initial guess value for x.
>
>Thank you.
>
>Alexei.

x:=.33
cos(x) = x   ==> 0.9460423435 = 0.33

\start
Date: Wed, 31 Dec 2003 03:45:47 -0500
From: Tim Daly
To: David Mentre
Subject: Re: [Axiom-math] documentation and the crystal
Cc: Gilbert Baumslag, Bill Page

>Usually, people have a top-down approach: they modelize a system
>abstractly (using more or less formal notations as UML or SDL) and then
>refine them through the actual code. But, in the case of Axiom, we need
>the reverse. We need to start from concrete objects (files, lines of
>source code) and add semantics to climb levels of abstraction. Of
>course, you follow different ladders, in the sense that understanding
>the compiler or the algebra would need different information and is
>structured differently, thus the different crystal facets of Tim.
>
>
>More concretely, I would propose the following approach:
>
> 1. start from parsers for the src/ directory. Parse directory structure
>    and each file, categorize them (boot, lisp, spad, ...) and construct
>    basic abstractions (list of lisp and boot functions; list of spad
>    categories, domains and operators; ...)

If you look at the int/algebra directory you'll see directories called
NRLIBs (e.g. DHMATRIX.NRLIB) which contain several files. These files
are output from the spad compiler. The databases are built from information
in these files. We can enhance the spad compiler to include additional
information in forms that can be written into databases or semantic
networks or whatever we like. The current databases (*.daase) files are
random access files. There is a C program called asq which knows how to
read these databases and answer queries. (src/etc/asq.c)

>
> 2. from information extracted in step 1, construct one or several
>    representation (knowledge graph for example) with the found
>    semantics (name and body of a function for example), probably using
>    a standard technology as W3C semantic web

I'll have to look at Bill's references to see what technology they are
using for the semantic web. There are several dozen ways of doing this
and almost all of them are not "well-founded" in any mathematical sense.
KREP is well founded. That is, when you build a semantic network one thing
you want to do is put a concept into the network "in the appropriate place".
Most of these systems have heuristics for doing this. KREP has a logically
sound, well-defined predicate called subsumption for comparing two concepts
and deciding if one subsumes the other. In KREP a new concept is put in
the semantic network by starting at the top and using the subsumption
predicate to push the concept down until it hits "the right place".

Semantic networks based on "is-a" links, or "kind-of" links and other
such random ideas are badly formed and suffer from a variety of logical
failings.

> 3. the "Axiom explorer" (Tim, Bill or I) is interested in a specific
>    "crystal facet" (e.g. the compiler). He builds another tool
>    (e.g. call-graph analyzer for lisp code) which in turn is used to
>    construct a new knowledge graph, concretized in additional
>    information in the W3C semantic web.
>
> 4. Within the abstraction level build in step 4 (or 2), the "Axiom
>    explorer" adds its own knowldege (e.g. this set of functions is used
>    to parse Spad code, this other set is used for type analysis, ...)
>    to the semantic web.
>
> 5. using previous abstraction levels and probably building a new tool,
>    the "Axiom explorer" iterates, climbing abstraction levels, until he
>    reaches his own goal.
>
>Repeat steps 1 to 5 with enough people to cover Axiom from A (Algebra)
>to Z (zerodim). 
>
>Ok. I might be a bit optimistic and what I have said might appear more
>than abstract, but this is the current state of might thoughts. :) 
>
>Even if you do not like above ideas, I think following "principles" are
>needed for a documentation system for Axiom (principles already
>formulated by Tim is his first email):
>
> o separation principle: in engineering, people separate a complex issue
>   in _independant_ sub-issues to be able to understand them and solve
>   them independently. We probably need to deconstruct Axiom in
>   independent (or a least loosely connected) sub-systems to be able to
>   understand them (i.e. the different crystal facets of Tim);

I stole this principle from the unix world. use simple tools (asq, makefile)
and gang them together to make more complex tools. Thus I see asq-style
programs triggered by clicking on a facet to answer a question. Some facets
might be backed by things like texmacs, xdvi, or latex.

>
> o "build on giant shoulders" principle: we need a way to reuse
>   knowledge from other "crystal facets". For example, I would use Tim
>   knowledge of the internals of Axiom to understand how the compiler
>   compiles a given portion of the algebra;

5'8" hardly qualifies as a giant :-) 

>
> o automation principle: Axiom is too big to add information manually on
>   each function, each object, etc. We need tools to annotate a set of
>   objects given a selection criteria (e.g. all operators in this Spad
>   domain). 

I'd like to automate as much as possible but the automation curve
gets steep in a hurry. At least for the algebra code we already have
tools in place. If we can get to where we can draw a lattice in the
next 6 months I should have the data (both hand and machine generated)
for the algebra.

>
>New year wish: I'll try to write and "show you the code" for above
>ideas. :) My own todo for this subproject of Axiom would be:
>
> - learn more about W3C Semantic Web (thank you Bill for the pointer)
>
> - find or write tools to manipulate Semantic Web (it might be Emacs
>   with a proper mode or a more elaborated graphical tool)
>
> - apply above approach, starting from directory structure in src/
>   directory of Axiom.
>
> - from this first experiment, think about what would need to be
>   "standardized", like common dictionnary or vocabulary, etc. Beyond
>   usual technological issues, I think this point is one fo the harder
>   point. How to build a set of knowledge that will be still useful in
>   30 years from now?

>By the way, does anybody knows about a library to *draw* and manipulate
>arbitary graphs (in Common Lisp, C++, ML, ...) in a user interface? I
>know about DOT or VCG but they dot not match my needs: I would like to
>draw a graph, know when the user click on a graph node or edge and react
>accordingly. Does anybody know where I could find such an already made
>tool? Any knowledge of a browser for the W3C semantic web?

There used to be a tool that would allow you to move around in a graph.
It was used to give a clever desktop. As you moved over portions of the
desktop the things nearest the mouse pointer got larger and were centered.
I can't remember what it was called but I thing M$ used a similar idea
in XP with the toolbar.

I'm going to try to get ahold of a tool that runs in common lisp and
gives access to the gtk bindings. We can use it to prototype some basic
functionality (like opening an empty window, drawing a graph in the window,
embedding texmacs in the window, etc. The developer will be back on the
5th and I'll look into it further at that time.

\start
Date: Wed, 31 Dec 2003 12:58:33 +0100
From: David Mentre
To: list
Subject: More random thoughts on the crystal: goals & technologies
Cc: Gilbert Baumslag

Hello,

Thank you Tim, Bertfried and Bill for detailed replies. It makes me
realize that, despite I have said before, we have slighlty different
short term objectives. I try to summarize the discussion up to now.

1) Goals

 A) Short and Mid term goals 

 As far as I have understood, we have two short/mid term goals:

 a) (Bill, Tim, Bertfried) Document the algebra to have a kind of
    reference manual for the Axiom programmer and researcher. Bertfried
    has detailed the kind of Type browser he would like to have in the
    short term
    (http://mail.nongnu.org/archive/html/axiom-developer/2003-12/msg00042.html).

 b) (David) Document the compiler internals. I would like to start by
    documenting the usage for the various files found in the src/
    directory.


 B) Long term goals

 We all want a system to navigate into _structured_ information that
 document the whole Axiom system. We all agree that you have different
 views on such a big system and, depending on your current work, we want
 to look more specifically at a subset of those views. Tim is using the
 idea of a Crystal, with several facets. Tim has detailed some facets
 examples:
 http://mail.nongnu.org/archive/html/axiom-developer/2003-12/msg00056.html 

 Now the hard part: which structure? How to document? How to explore?
 What underlying technology?

 Regarding structure, I have proposed to use a graph like approach. Tim
 has expressed interest for a Knowledge Network with a sound underlying
 technology (KROPS). Bertfried has proposed to use a Matroid
 approach. Tim and Bertfried objections is that we need a _structured_
 way of representing semantics in Axiom. I agree with them.


 Regarding ways how to add information to current Axiom, several
 approaches have been proposed by Tim
 (http://mail.nongnu.org/archive/html/axiom-developer/2003-12/msg00069.html).
 In summary:
  - COLLECT COMPILER INFORMATION: use compiler knowledge to produce
    indexes and other useful information
  - LATEX STRUCTURE INFORMATION: add manual information (issue:
    difficult to parse using a computer). Already used by Tim to
    document the build process
  - SPECIAL INFORMATION STRUCTURE: building "information sources" on
    specific topics. Several sources for different purposes can be made
  - GENERAL MATHEMATICS STRUCTURE: impose mathematical structures on
    facets 
  - PERSONAL FACETS: the user builds its own vision of the system, using
    information available in other users' system (it reminds me of
    SmallTalk Environment)
 

 From those several ideas, I deduce that we will probably need several
 information sources (i.e. like databases but possibly with a different
 model that the relational model). Those information sources should be
 able to be parsed by a computer, but sometimes it won't be possible at
 full extent (e.g. the PhD included in DHmatrix: we can extract some
 literate programming structure but the full meaning of the english
 text cannot be understood by a computer).

 We will also need several tools (Unix approach) to pick up information
 is those several sources (and probably build other sources). The main
 issue is to not use too many technologies, languages, etc.


2) Technologies

 a) W3C Semantic Web

 Bill as proposed to consider the W3C Semantic Web, which is build
 around XML technologies like RDF (Resource Description Framework) and
 OWL (Web Ontology Language). Those technologies are probably not well
 enough structured (in the sense that Tim and Bertfried want to
 structure semantics added to Axiom) but they form a standard that we
 could build upon. Even using unstructured RDF, we could _impose_ our
 own structure (e.g. Bertfried's Matroid) on our RDF files.

 Another important point in that they are build from the ground-up to be
 distributed and reusable by any user.

 Those technologies are not very easy to use (e.g. cumbersome syntax)
 but we could use them as a common file format to store our knowledge on
 Axiom (Tim's information sources). It would allow use to build on
 existing tool and infrastructure, to not reinvent the wheel and to
 maybe to gather additional developers and community interest.

 For example, it exists a lot of research on ways to browse RDF files
 (like MIT Haystack if I remember correctly).


 b) Ad-hoc information sources

 Tim as suggested that we instrument the compiler to produce additionnal
 information used to build information source. Using current tool (ASQ),
 we could query those information. The advantage is that the technology
 is already there. The disavantage is that the file format is
 proprietary to Axiom, so we will need to build bridges with other tools
 and technologies (browsers, query languages, ...). Bu I agree with Tim
 that it can be used (1) as a first step and (2) as a proof of concept.


 c) literate programming

 Noweb is useful as a first step to add information onto existing
 code. However, we need to find a way to categorize this information and
 build relations. We need a multi-dimensional book. Should we
 standardize the naming of pamphlets (<<algebra:typing:...>>) or should
 we add another technology (RDF?)  above or below code and documentation
 chunks?


 d) "browsers"

 "Browsers" (in a wide sense, i.e. more than just showing hyper-links)
 should allow users:

  - to add information is both a distributed (simultaneously in France,
    Germany, Canada and US) and structured way

  - to "navigate" into this information, using both graphical approaches
    (e.g. follow hyperlinks) and computational approaches (e.g. query
    languages)

  They should be build on existing tools and technology (TeXmacs?
  Haystack? Web browsers & HTML?).


 e) Knowledge Network

  Tim has expressed interest for a tool like KROPS
  (http://mail.nongnu.org/archive/html/axiom-developer/2003-12/msg00071.html).
  It would allow users to put semantics in the computer in a sound
  way. Possible issues: is it difficult to build such a system? How to
  interconnect it with current and future information sources?
 
 

3) What to do?

 I think our community has setup the goals the short term goals:

 - fix our current bugs (just a reminder ;), have a documentation and
   release our first public release of Axiom

 - build a type browser as Bertfried request

 - build a usable lattice of the algebra 

 - (for myself, I would like to also document the files in src/)


 After those steps, we could make a point (a kind of workshop), draw
 conclusion and ways towards the next step.


 To decide on the technology to use, we need a consensus. Personnaly, I
 am open to any technology, programming language (please avoid Java :),
 etc. For the short term, we could probably consider using existing
 information sources and tools (asq) and then consider a migration path
 towards more standardized file format (RDF?).

 What is your opinion?





