UDF toolkit TODO list for release version 4

On slowaris (0.3nb2) :
	<Underhand> In file included from udfdump.c:39:
	<Underhand> udf_bswap.h:40: machine/endian.h: No such file or directory
	<Underhand> udf_bswap.h:45: machine/bswap.h: No such file or directory
	<Underhand> udf_bswap.h:46: sys/bswap.h: No such file or directory


Important fixes first to have :
	- implement VRS caching and writeout (for recordables)

On the roll (random order) :
----------------------------
udfclient :
	- implement `chmod'/`chown' etc. in udfclient <-- using VOP_SETATTR
	- implement bad blocks detection for formatting; maybe offer an option
	  to re-use them on a new format?
	- `readline' support for easier directory/file selection
	- ....

newfs_udf :
	- implement allocation of sparable-table in newfs
	- implement formatting of UDF 2.50 media
	- ....

general :
	- start with a very basic `fsck_udf' ?
	- ....

internal:
	- implement write access to discs formatted with UDF 2.50
	- modify device type classification to match SCSI device types
	- important: sizeof(struct dirent) doesn't nessisarily include the
	  space for the name!!! better create a constant size in d_reclen that
	  is derived from sizeof(dirent) + NAME_MAX or _DIRENT_RECLEN.
	  Dragonfly has no d_reclen though its not used elsewhere.
	- redo VRS completely to have one mountpoint/disc but with multiple
	  partition support offcource <-- desireable for CD/DVD support.  For
	  discs means that partitioning is not done by UDF.
	- simplify udf_nodes a lot by not interpreting the allocations but
	  work directly on them.
	- implement ISO Volume Recognition Sequence check/writeout
	- implement disc renaming
	- implement writable y/n determination more rigorously
	- Implement writing to CD-R/DVD+R's :
		- implement allocation
		- implement writing back of virtual sectors
		- implement VAT writeback
	- speed up lookup by remembering where we were on this udf_node
	- implement file/dir operation rename outside directory
	- implement udf_filepart_realloc() stuff for write errors.  Note this
	  is not nessisary anymore; most severe problem is with write problems
	  with nodes; then we need to use the sparable parts.  NOTE: this is
	  easy to implement.
	- dumping of extended attributes by also dumping extended attributes
	- When creating a node, remember the extended attributes ICB (the
	  sector blob address). Also write it back.
	- implement blocksize extended attributes from EFE.
	- implement streamdir support (!) esp. important when deleting.
	- Read in embedded extended attributes, preprocess relevant ones,
	  compress and remember. Write out the extended attributes when
	  writing out node.
	- implement lazy writing for non descriptors by checking if its
	  changed and if not so skip it.
	- auto-sparable-files : when writing all zero blocks, mark the space
	  as non allocated and free the buffer's claim on disc allocation.
	  file.
	- add support for incremental formatting of CD-RW/DVD+-RW ?
	- maybe rename `split' into a more describing primitive? i.e. that it
	  can also extent?
	- rename allocentry and/or allocentry_queue ....
	- ....

0.6+ :
	- if link count == 1, a file/dir node can be MOVED around since all
	  parents are KNOWN :) so reallocating nodes on disc is possible since
	  we know there are no hidden links to it thus making our own variant
	  of `softdep' by keeping a newer `shadow' tree on disc and swapping
	  trees when the modifications are all written out. If something
	  really fails it is guaranteed that : 1) all dirs/files are at a
	  previous consistent state with a lot of logical blocks recounting to
	  do ... but fsck_udf will have to check & fid that allways since its
	  left marked open. 2) in the most severe case a broken directory
	  entry in a dirstream is found. NOTE that shadow trees ought to be
	  written back in the order they are created.

Open choices:

Interesting possible speed/complexity reduction:
	- implement namei cache... this could speed up things
	- if namei cache fails; maybe back to complete directory readins?

Philosophical:
	- write error callback strategy thinkout/brainstorming; what to do
	  when and under what cercumstances.
	- volume stuff management rethink, how to allow additions/removals of
	  discs/partitions, etc. and what is sane to support. Also think about
	  the drive/session/logvol/mountpoint differentiation.
	- ....

DONE :
	- add byteswap support for PVR's
	- fix cd_discect to be more clean
	- big change: use queue.h instead of linked lists and hash_table
	- implement inode cache in lookup
	- implement write_session sector, write_session_bytes_uio
	- implement write QUEUE
	- implement session cache writeout function
	- implement logical cache for recordables (read/write_log_vol...).
	  (logical cache is not block oriented)
	- implement writing functions for session and logvol descriptors
	- implement sparse file writing support()
	- implement sparse file reading support()
	- implement allocation descriptor queue comprimation; executed when?
	  well at least before the descriptor gets written back. Say also
	  before each write action?
	- implement keeping track of descriptor addresses when reading in file
	  descriptor sequences in the process of creating udf_node's
	- implement write_file_part()
	- implement writing back of file descriptors
	- reorder free/used space reading in
	- implement rw/locking on udf_allocentry's queue <- done by locking
	  node
	- implement free/used space management (udf_allocate_free_logblks())
	- implement free/used space descriptor write back
	- Fix write speed indication in udfclient
	- Fix not existing file writeout bug in udfclient
	- Fix: pass on descriptor version when building new desc tags
	  (newfs_udf mostly).
	- Implement embedded data in descriptor writing; conversion from one
	  type to another is messy :(
	- Implement file truncation with free space giveback
	- Fix VOP_TRUNC()/VOP_WRITE() interaction... late writebacks can cause
	  trouble when VOP_TRUNC is called early; when shrinking first call
	  `sync' ? <--- implemented but is it enough?
	- implement file/dir delete 
	- fix ISDOTDOT lookup in VOP_lookup
	- Implement and test harddisc device read/write
	- implement create file/dir
	- implement file/dir reference counters
	- Implement recursive `put' in udfclient just as the recursive `get'
	  does.
	- Fix extra unnessisary trunc()'s
	- investigate remove thread and transform `queue kick' to the writeout
	  function. REJECTED: with the two pthread condition variables
	  sheduling is much aproved and this would be a step back.
	- investigate converting virtual write queue cache merge with ICB
	  holding for recordables; making it one big array? i.e. 64 sectors?
	  expensive though :( Would simplify logvol virt. write queue a lot.
	  REJECTED: new scheme for metadata makes this unnessisary and
	  unwanted.
	- start of new caching algorithm... complete rewrite.
	- test and use udf_cut_allocentry_queue as primitive; it ought to fix
	  most problems with cutting queue
	- `waiting-to-be-written' parts should not be marked BUSY but marked
	  ALLOCATED. Non-written but alloced stuff ought to be marked FREED.
	  This ensures that readers will allways read the correct information.
	  The write-count will ensure syncability. This also eliminates
	  write-thread modifying of alloc-entries other then when write-errors
	  occure.
	- REJECTED reimplement read/write cache as tailq? (one cache tailq
	  with a seperate subset of writeq as currently done)
	- merge maximum blocksize extending from `truncate udf node' into
	  `split udf node' code and remove code from truncate.
	- split read-in and write-out phases; when the writing is busy, no
	  reading is allowed and should be waiting. Note that the writing will
	  be significantly faster now that all missing sectors are allready
	  taken care for and so no switch-over of the drive is needed.
	- create session write thread mechanism again with (most importantly)
	  fill-up reading.... note that this most likely eliminates the
	  NOREADIN flag.
	- implement Logvol integrity descriptor reading/writing including
	  unique file numbering; its getting in the way.
	- implement file/dir delete recursion in udfclient
	- fix Linux sg support for 2.6 kernel
	- use  PRIu32 and PRIu64  inttypes.h
	- FreeBSD SCSI/CAM support
	- add format utility for CD-RW discs
	- when `sync' is called, not only kick the disc that is holding the
	  logvol but all discs.
	- ICB clustering
	- Implement `lazy' free space management? i.e. marking as
	  allocated/not written and later after write marking written (2) ?
	- allways start a new file with internal allocation. Implement
	  automatic switching of alloc entries->internal (part of trunc?).
	  Implemented as auto-switching on writeout; starts with a long alloc
	  to prevent space allocation problems later on.
	- implement non 2048 bytes sectors; udfclient can now read and write
	  discs formatted with any blocksize of 512. Also not fully tested
	  implementation of lb_size being a multiple of sector_size.
	- don't be strict on disc block allocation for metadata; for data only.
	- implement file/dir rename
	- implement logical volume integrity sequence supporting multiple
	  chains of integrity blocks to release disc blocks of stress.
	- REJECTED read and write descriptors at offsets so one can easily use
	  the descriptor read/write functions for directory entries.
	- record the right sector numbers in the directory entries.
	  Implemented by keeping track of what kind of data it is and at the
	  writeout-logvol sector level (!) patch up the lb numbers.
	- implement close/open session (!) maybe earlier?
	- add support for formatting DVD+RW/DVD-RW ?  NOTE: create an extra
	  cdrw_format clone dvdrw_format ?
	- implement dumping of extra attributes on file descriptors
	- implement userID, groupID and access rights preservation
	- implement timestamping access, modify, attr times
	- implement read-only access to discs formatted with UDF 2.50

