ANNOUNCE - Muldis D v0.27.0 released

All,

I am pleased to announce the release of Muldis D
official/unembraced 
version 0.27.0 on CPAN.  You can see it now, with nicely
HTMLized 
documentation, at:

   http://search.c
pan.org/dist/Muldis-D/

--------

Also, the reference implementation of Muldis D is now named
Muldis Rosetta, 
not Muldis DB, and it is currently at CPAN version 0.7.0;
but you can 
ignore that for now as it doesn't do anything yet; it will
be updated to 
work within a few weeks probably.

But today's email is all about the Muldis D language spec
itself.

--------

The Muldis D language spec had a large number of
improvements and changes 
since my last announcement email about version 0.15.0 on Dec
31st, and I 
will summarize some of the important ones below:

1.  Split up the Core.pod file, which had most of the system
defined types 
and routine definitions, into about 13 files, where 4 are
named 
lib/Muldis/D/Core/*.pod and 9 are named
lib/Muldis/D/Ext/*.pod.  So the 
collection of core entities has been reduced to a reasonable
minimum, one 
that is relatively easy to implement/manage, while still
being a superset 
of what is needed to satisfy "The Third
Manifesto", meaning a complete 
truly relational DBMS (and meets an ORDBMS definition also).
 The core 
includes the scalar types [boolean, integer, text, blob],
plus tuple and 
relation and database constructors, plus a full system
catalog by which 
users can define arbitrarily complex types and routines
(well, type 
constructors are still missing, but they would be in the
core), plus a 
standard complement of boolean and relational algebra
operators, plus the 
equality test and assignment ops for all types, and
store/retrieve ops. 
The collection of extension entities includes the scalar
types 
[rational/float, temporals, spatials], plus their operators,
plus most of 
the standard operators for [integer, text, blob], plus
generic ordinal 
operators, plus more collection operators, plus (to come)
pattern matching 
/ replacing operators.  Essentially we've moved out of the
core whatever we 
could without it getting stupid.  And the docs should be
much easier to read.

2.  The actual Muldis D standard grammars/dialects have been
further 
formalized and fleshed out, as well as renamed into 
lib/Muldis/D/Dialect/*.pod.  There are currently 2 standard
dialects, whose 
names are "PTMD_Tiny" (Tiny Plain Text Muldis D)
and "HDMD_Perl_Tiny" (Tiny 
Perl Hosted Data Muldis D).  They are named Tiny because
their grammars are 
very simple and so a parser grammar capable of handling them
could be tiny; 
in fact, PTMD_Tiny includes an actual grammar as Perl 6
rules which is less than 7K in size.  Note that Muldis D is
a homoiconic 
language (one of its most important features in fact), so
the fundamental 
way to write Muldis D code is to write Muldis D data, and
you compile that 
code by assigning said data to catalog dbvars, a process
analogous to 
inserting records into the INFORMATION_SCHEMA of SQL, as if
SQL even 
allowed that.  Therefore, like Perl or Lisp or various
languages, you can 
make your own language dialect as it suits you, for writing
your own code 
in.  The standard language comes with the two Tiny dialects,
which are 
hopefully good and terse enough for many people to just use
directly, so 
you can just start writing in it right away, but if people
find these too 
verbose, well they are empowered for other options. 
PTMD_Tiny is for 
people that want to write and run Muldis D directly like a
normal Turing 
Complete programming language (which it is, though
specialized for 
databases); I foresee a Parrot compiler for it some time, or
a shell like 
SQL*PLUS et al.  HDMD_Perl_Tiny is for people that want to
write Muldis D 
code using trees of Perl Arrays, like they do with
SQL::Abstract et al, and 
this option is best when you're dynamically building
database commands in 
Perl at runtime.  For static database commands (which still
are 
parameterized), you can use both dialects in Perl programs. 
The 2 Tiny 
dialects map to each other 1:1, more or less.  They are
essentially complete.

NOTE:  ANOTHER EMAIL WILL SOON FOLLOW THIS ONE WHICH
CONTAINS SOME CODE 
SAMPLES IN ACTUAL "TINY" MULDIS D CODE, AND NOT
JUST PSEUDO-CODE.

3.  The standard rational numeric type and its literal
syntax make it 
capable of exactly representing any rational number, in any
numeric base 
and in any magnitude (limited only by system memory or
resource limiters or 
subtype constraints).  Like the standard integers, these are
"big".  This 
rational type also includes the functionality of
floating-point types but 
that all digits are significant.  All Muldis D numerics are
also 
radix-independent, so base 2 or base 10 or base N you don't
lose accuracy 
anywhere except when you explicitly choose to.  Note that
the standard 
numerics don't include special values like infinities or
NaNs, but some 
extension could provide those.  And Complex et al doesn't
exist yet.

4.  Although Muldis D explicitly is 2-valued-logic
(true|false only) and 
not N-valued-logic (true|false|missing|inapplicable|...),
the collection 
types Maybe|Nothing|Single and their special syntax let you
design your 
database and code as if a 3+VL NULL actually existed.  To
explain, the 
Muldis D analogy of a SQL nullable value is a set value that
can contain 
either zero or one element; if the set is empty, the value
is NULL, if it 
has an element, the value is NOT NULL, and the set's element
is the value; 
in contrast, the Muldis D analogy of a SQL not-nullable
value is simply the 
value that was the non-empty set element, but without having
to be wrapped 
up in a set.  So Maybe is a set with 0 or 1 element, Nothing
is a set with 
no elements, and Single is a set with 1 element; the latter
2 are 
complementary subtypes of the first.

5.  Formalized the process for naming versions of the Muldis
D language 
spec (and formatting those names), taking into account
multiple authorities 
and dialects et al, like Perl 6 long package names but
somewhat different. 
  See the VERSIONING pod section in D.pm for that.  Any
Muldis D code 
should begin with a statement stating explicitly what 
authority+version+dialect it is, so to prevent ambiguity by
machine or 
human readers.  This is analogous to the "use
5.008" et al that Perl files 
often start with.  The main purpose of this is to
future-proof Muldis D 
code, so that either I or others can be free to modify the
language in 
backwards-incompatible ways, but interpreters for the new
versions can 
still work without ambiguity problems with code in older
versions if they 
so choose.  Also a good guard against the consequences of
"embrace and 
extend and extinguish" by third parties.

6.  Overhauled the namespace hierarchy for in-DBMS entities.
 So standard 
language types and routines are in the sys.std.* namespace
(note that most 
the docs on builtins don't reflect the 'std' parts yet, but
will), while 
implementation-specific extensions are in the sys.imp.*
namespace, the 
latter being further subdivided by authority.  This feature
permits the 
standard designer to add or rename things without worrying
about stepping 
on some implementers add-ons, and different implementers
likewise with each 
other.  Also the system catalog is no longer under cat.*,
but is rather 
split up and located by what it describes, such as at
sys.cat 
(system-defined entities), fed.cat (details of all connected

depots|databases), dep.cat (details of context-current
depot|database), 
sdp.cat (details of context-current subdepot|schema),
pkg.cat (details of 
context-current package, like an Oracle package). 
Similarly, the 
context-local types and routines are under eg dep.lib, and
the user's data 
relvars|tables under eg dep.data.

7.  The system catalog, and Tiny dialects, now can capture
and store code 
comments along with the code itself.  So translating code
from one dialect 
to another, or to/from SQL, can carry across the code
comments.  This 
feature, storing comments as normal AST elements, is also
part of the 
FoxPro language as I recall, though I don't know if any
other languages do 
it; SQL only accomplishes that because it stores SQL
routines as source 
code text, not the same thing as we do.

8.  Rewrote the catalogs / catalog types for defining
routines (functions, 
updaters, procedures), and added catalog types for defining
whole depots, 
packages, namespaces, etc.  Still to do, rewrite/add ability
to define new 
types (next release I hope), and by extension, database
relvars|tables and 
constraints.

9.  Added bushels more relational and other operators.

10.  Added notice that the word MULDIS is a trademark of
mine (invented by 
me as a contraction of MULTIVERSE OF DISCOURSE, that
specific term also 
invented by me), and also that I offer commercial support et
al 
specifically by way of the business Muldis Data Systems.

See the "Changes" file ( 
http://search.cpan.org/src/DUNCAND/Muldis-D-0.27.0/Cha
nges ) for more 
details of what changed for the last several releases.

--------

Still to do:

1.  Overhaul and expand the catalog types for defining data
types, whole 
database constraints, views, triggers, etc.  This one in a
few days hopefully.

2.  Overhaul and expand relational operators for sorting and
quota queries; 
that is, analogues to SQL's ORDER BY and LIMIT tasks, as
well as 
order-significant aggregate operations on relation
attributes (eg, catenate).

3.  Define more flow control operators like IF/ELSE,
GIVEN/WHEN, etc.

4.  Flesh out and add system service operators like random
numbers or 
current dates or simple generic I/O.

5.  Add operators for pattern matching, search/replace etc,
eg both regular 
expressions and SQL's "LIKE" (not just "is
substring" that already exists).

6.  Overhaul date and time types, add operators for them.

--------

Thank you. -- Darren Duncan

If I want to follow development of your project,
I can subscribe to its mailing lists,
so I don't know why these announcements are
always posted to the Bricolage development list.
Why not wait until it's actually useable
and has some relevance to Bricolage,
then show us its magical powers by
reimplementing Bricolage with it?

On Mon, 28 Apr 2008, Darren Duncan wrote:
> All,
>
> I am pleased to announce the release of Muldis D
official/unembraced version 
> 0.27.0 on CPAN.  You can see it now, with nicely
HTMLized documentation, at:
>
>  http://search.c
pan.org/dist/Muldis-D/
>
> --------
>
> Also, the reference implementation of Muldis D is now
named Muldis Rosetta, 
> not Muldis DB, and it is currently at CPAN version
0.7.0; but you can ignore 
> that for now as it doesn't do anything yet; it will be
updated to work within 
> a few weeks probably.
>
> But today's email is all about the Muldis D language
spec itself.
>
> --------
>
> The Muldis D language spec had a large number of
improvements and changes 
> since my last announcement email about version 0.15.0
on Dec 31st, and I will 
> summarize some of the important ones below:
>
> 1.  Split up the Core.pod file, which had most of the
system defined types 
> and routine definitions, into about 13 files, where 4
are named 
> lib/Muldis/D/Core/*.pod and 9 are named
lib/Muldis/D/Ext/*.pod.  So the 
> collection of core entities has been reduced to a
reasonable minimum, one 
> that is relatively easy to implement/manage, while
still being a superset of 
> what is needed to satisfy "The Third
Manifesto", meaning a complete truly 
> relational DBMS (and meets an ORDBMS definition also). 
The core includes the 
> scalar types [boolean, integer, text, blob], plus tuple
and relation and 
> database constructors, plus a full system catalog by
which users can define 
> arbitrarily complex types and routines (well, type
constructors are still 
> missing, but they would be in the core), plus a
standard complement of 
> boolean and relational algebra operators, plus the
equality test and 
> assignment ops for all types, and store/retrieve ops.
The collection of 
> extension entities includes the scalar types
[rational/float, temporals, 
> spatials], plus their operators, plus most of the
standard operators for 
> [integer, text, blob], plus generic ordinal operators,
plus more collection 
> operators, plus (to come) pattern matching / replacing
operators. 
> Essentially we've moved out of the core whatever we
could without it getting 
> stupid.  And the docs should be much easier to read.
>
> 2.  The actual Muldis D standard grammars/dialects have
been further 
> formalized and fleshed out, as well as renamed into 
> lib/Muldis/D/Dialect/*.pod.  There are currently 2
standard dialects, whose 
> names are "PTMD_Tiny" (Tiny Plain Text Muldis
D) and "HDMD_Perl_Tiny" (Tiny 
> Perl Hosted Data Muldis D).  They are named Tiny
because their grammars are 
> very simple and so a parser grammar capable of handling
them could be tiny; 
> in fact, PTMD_Tiny includes an actual grammar as Perl
6
> rules which is less than 7K in size.  Note that Muldis
D is a homoiconic 
> language (one of its most important features in fact),
so the fundamental way 
> to write Muldis D code is to write Muldis D data, and
you compile that code 
> by assigning said data to catalog dbvars, a process
analogous to inserting 
> records into the INFORMATION_SCHEMA of SQL, as if SQL
even allowed that. 
> Therefore, like Perl or Lisp or various languages, you
can make your own 
> language dialect as it suits you, for writing your own
code in.  The standard 
> language comes with the two Tiny dialects, which are
hopefully good and terse 
> enough for many people to just use directly, so you can
just start writing in 
> it right away, but if people find these too verbose,
well they are empowered 
> for other options.  PTMD_Tiny is for people that want
to write and run Muldis 
> D directly like a normal Turing Complete programming
language (which it is, 
> though specialized for databases); I foresee a Parrot
compiler for it some 
> time, or a shell like SQL*PLUS et al.  HDMD_Perl_Tiny
is for people that want 
> to write Muldis D code using trees of Perl Arrays, like
they do with 
> SQL::Abstract et al, and this option is best when
you're dynamically building 
> database commands in Perl at runtime.  For static
database commands (which 
> still are parameterized), you can use both dialects in
Perl programs.  The 2 
> Tiny dialects map to each other 1:1, more or less. 
They are essentially 
> complete.
>
> NOTE:  ANOTHER EMAIL WILL SOON FOLLOW THIS ONE WHICH
CONTAINS SOME CODE 
> SAMPLES IN ACTUAL "TINY" MULDIS D CODE, AND
NOT JUST PSEUDO-CODE.
>
> 3.  The standard rational numeric type and its literal
syntax make it capable 
> of exactly representing any rational number, in any
numeric base and in any 
> magnitude (limited only by system memory or resource
limiters or subtype 
> constraints).  Like the standard integers, these are
"big".  This rational 
> type also includes the functionality of floating-point
types but that all 
> digits are significant.  All Muldis D numerics are also
radix-independent, so 
> base 2 or base 10 or base N you don't lose accuracy
anywhere except when you 
> explicitly choose to.  Note that the standard numerics
don't include special 
> values like infinities or NaNs, but some extension
could provide those.  And 
> Complex et al doesn't exist yet.
>
> 4.  Although Muldis D explicitly is 2-valued-logic
(true|false only) and not 
> N-valued-logic (true|false|missing|inapplicable|...),
the collection types 
> Maybe|Nothing|Single and their special syntax let you
design your database 
> and code as if a 3+VL NULL actually existed.  To
explain, the Muldis D 
> analogy of a SQL nullable value is a set value that can
contain either zero 
> or one element; if the set is empty, the value is NULL,
if it has an element, 
> the value is NOT NULL, and the set's element is the
value; in contrast, the 
> Muldis D analogy of a SQL not-nullable value is simply
the value that was the 
> non-empty set element, but without having to be wrapped
up in a set.  So 
> Maybe is a set with 0 or 1 element, Nothing is a set
with no elements, and 
> Single is a set with 1 element; the latter 2 are
complementary subtypes of 
> the first.
>
> 5.  Formalized the process for naming versions of the
Muldis D language spec 
> (and formatting those names), taking into account
multiple authorities and 
> dialects et al, like Perl 6 long package names but
somewhat different.  See 
> the VERSIONING pod section in D.pm for that.  Any
Muldis D code should begin 
> with a statement stating explicitly what
authority+version+dialect it is, so 
> to prevent ambiguity by machine or human readers.  This
is analogous to the 
> "use 5.008" et al that Perl files often start
with.  The main purpose of this 
> is to future-proof Muldis D code, so that either I or
others can be free to 
> modify the language in backwards-incompatible ways, but
interpreters for the 
> new versions can still work without ambiguity problems
with code in older 
> versions if they so choose.  Also a good guard against
the consequences of 
> "embrace and extend and extinguish" by third
parties.
>
> 6.  Overhauled the namespace hierarchy for in-DBMS
entities.  So standard 
> language types and routines are in the sys.std.*
namespace (note that most 
> the docs on builtins don't reflect the 'std' parts yet,
but will), while 
> implementation-specific extensions are in the sys.imp.*
namespace, the latter 
> being further subdivided by authority.  This feature
permits the standard 
> designer to add or rename things without worrying about
stepping on some 
> implementers add-ons, and different implementers
likewise with each other. 
> Also the system catalog is no longer under cat.*, but
is rather split up and 
> located by what it describes, such as at sys.cat
(system-defined entities), 
> fed.cat (details of all connected depots|databases),
dep.cat (details of 
> context-current depot|database), sdp.cat (details of
context-current 
> subdepot|schema), pkg.cat (details of context-current
package, like an Oracle 
> package).  Similarly, the context-local types and
routines are under eg 
> dep.lib, and the user's data relvars|tables under eg
dep.data.
>
> 7.  The system catalog, and Tiny dialects, now can
capture and store code 
> comments along with the code itself.  So translating
code from one dialect to 
> another, or to/from SQL, can carry across the code
comments.  This feature, 
> storing comments as normal AST elements, is also part
of the FoxPro language 
> as I recall, though I don't know if any other languages
do it; SQL only 
> accomplishes that because it stores SQL routines as
source code text, not the 
> same thing as we do.
>
> 8.  Rewrote the catalogs / catalog types for defining
routines (functions, 
> updaters, procedures), and added catalog types for
defining whole depots, 
> packages, namespaces, etc.  Still to do, rewrite/add
ability to define new 
> types (next release I hope), and by extension, database
relvars|tables and 
> constraints.
>
> 9.  Added bushels more relational and other operators.
>
> 10.  Added notice that the word MULDIS is a trademark
of mine (invented by me 
> as a contraction of MULTIVERSE OF DISCOURSE, that
specific term also invented 
> by me), and also that I offer commercial support et al
specifically by way of 
> the business Muldis Data Systems.
>
> See the "Changes" file ( 
> http://search.cpan.org/src/DUNCAND/Muldis-D-0.27.0/Cha
nges ) for more details 
> of what changed for the last several releases.
>
> --------
>
> Still to do:
>
> 1.  Overhaul and expand the catalog types for defining
data types, whole 
> database constraints, views, triggers, etc.  This one
in a few days 
> hopefully.
>
> 2.  Overhaul and expand relational operators for
sorting and quota queries; 
> that is, analogues to SQL's ORDER BY and LIMIT tasks,
as well as 
> order-significant aggregate operations on relation
attributes (eg, catenate).
>
> 3.  Define more flow control operators like IF/ELSE,
GIVEN/WHEN, etc.
>
> 4.  Flesh out and add system service operators like
random numbers or current 
> dates or simple generic I/O.
>
> 5.  Add operators for pattern matching, search/replace
etc, eg both regular 
> expressions and SQL's "LIKE" (not just
"is substring" that already exists).
>
> 6.  Overhaul date and time types, add operators for
them.
>
> --------
>
> Thank you. -- Darren Duncan

Scott Lanning wrote:
> If I want to follow development of your project,
> I can subscribe to its mailing lists,
> so I don't know why these announcements are
> always posted to the Bricolage development list.
> Why not wait until it's actually useable
> and has some relevance to Bricolage,
> then show us its magical powers by
> reimplementing Bricolage with it?

The reason I was posting these announcements to the
Bricolage list is that 
I thought they were relevant to Bricolage.

Completion of my project is imminent, where I consider July
1 to be a hard 
deadline for it to have a working implementation (and June 1
a soft 
deadline for the same).  At that point, Bricolage is at the
top of my list 
for FLOSS applications to be ported to it, though honestly I
don't expect 
to be doing that task by myself, and if a Bricolage Camp
occurs at OSCON 
2008, then porting Bricolage to use my project will be the
main reason I am 
intending to participate in it.  Now since I'm hoping the
port will be done 
by several of us and not just myself, I figured that if you
know its broad 
progress, you can be in the position to study or critique it
as to how it 
may meet your needs, early enough such that designs can be
altered to 
accommodate you if necessary.

Also, several proponents of my project are on the Bric list
and I don't 
know if they are all on other lists where I announce.

Also, I'm keeping my announcements quite infrequent and not
wasting your 
time with details.

Having explained this, if anyone else, particularly David,
agrees that this 
list is not an appropriate place for recurring infrequent
posts on my 
project, then I will stop making them here, only to change
if it later 
becomes appropriate again.

Meanwhile as a hedge, I will not send to this list the
companion email I 
was planning for today showing some code samples, unless
someone explicitly 
says go ahead and/or that the announcements are indeed
appropriate.

-- Darren Duncan

Scott Lanning wrote:
> If I want to follow development of your project,
> I can subscribe to its mailing lists,
> so I don't know why these announcements are
> always posted to the Bricolage development list.
> Why not wait until it's actually useable
> and has some relevance to Bricolage,
> then show us its magical powers by
> reimplementing Bricolage with it?

Actually, on further thought, I completely agree.

So I will not make any more posts about Muldis D et al to
the Bricolage 
list; yesterday's was the last.

My posting these here is probably akin to posting when DBI
is updated or 
when something else that Bric uses or might use is updated.

So like those, my project won't be brought up here anymore
except 
specifically in Bricolage implementing discussions that
involve it.

Those who still want such announcements in isolation can
subscribe to one 
of the project's own lists.

-- Darren Duncan

On Apr 29, 2008, at 12:57, Darren Duncan wrote:

> Actually, on further thought, I completely agree.
>
> So I will not make any more posts about Muldis D et al
to the  
> Bricolage list; yesterday's was the last.
>
> My posting these here is probably akin to posting when
DBI is  
> updated or when something else that Bric uses or might
use is updated.
>
> So like those, my project won't be brought up here
anymore except  
> specifically in Bricolage implementing discussions that
involve it.
>
> Those who still want such announcements in isolation
can subscribe  
> to one of the project's own lists.

Thanks Darren.

So, are we having an OSCON camp or not?

David

David E. Wheeler wrote:
> So, are we having an OSCON camp or not?

I believe having an OSCON Bric Camp is a good idea, though
it should not 
conflict with PGDay on Jul 20th.  If it happens, I will
attend and work 
with people on Bric improvements.  I don't intend to
organize the event 
though, so a more central developer will need to do it.  If
there is no 
official Bric Camp event, we can still arrange to meet in a
more ad-hoc 
fashion, such as within the generic free OSCAMP event which
runs all day 
M-F(?). -- Darren Duncan

On Apr 29, 2008, at 16:53, Darren Duncan wrote:

> David E. Wheeler wrote:
>> So, are we having an OSCON camp or not?
>
> I believe having an OSCON Bric Camp is a good idea,
though it should  
> not conflict with PGDay on Jul 20th.  If it happens, I
will attend  
> and work with people on Bric improvements.  I don't
intend to  
> organize the event though, so a more central developer
will need to  
> do it.  If there is no official Bric Camp event, we can
still  
> arrange to meet in a more ad-hoc fashion, such as
within the generic  
> free OSCAMP event which runs all day M-F(?). -- Darren
Duncan

Yeah, I didn't assume you would organize it. My query was
directed to  
others.

Best,

David

On 29-Apr-08, at 9:54 PM, David E. Wheeler wrote:

> On Apr 29, 2008, at 16:53, Darren Duncan wrote:
>
>> David E. Wheeler wrote:
>>> So, are we having an OSCON camp or not?
>>
>> I believe having an OSCON Bric Camp is a good idea,
though it  
>> should not conflict with PGDay on Jul 20th.  If it
happens, I will  
>> attend and work with people on Bric improvements. 
I don't intend  
>> to organize the event though, so a more central
developer will need  
>> to do it.  If there is no official Bric Camp event,
we can still  
>> arrange to meet in a more ad-hoc fashion, such as
within the  
>> generic free OSCAMP event which runs all day
M-F(?). -- Darren Duncan
>
> Yeah, I didn't assume you would organize it. My query
was directed  
> to others.


I've got another announcement about it ready to go out in
the April  
Output Channel. Still very little traction from the looks of
it:
http://wiki.bricolage.cc/bin/view/Bric/BricolageCamp2008




--
Phillip Smith,
Simplifier of Technology
Community Bandwidth
http://www.community
bandwidth.ca