Re: Recent changes in Range#step behavior

Hi,

In message "Re: Recent changes in Range#step
behavior"
    on Thu, 27 Mar 2008 02:25:54 +0900, Dave Thomas
<davepragprog.com> writes:

|If member? is allowed to use non-discrete comparisons for
integers,  
|then it should probably also do them for ranges of strings.
Right now
|
|('a'..'z').member?('aa')
|
|returns false, because 'aa' is not in the set 'a',
'b'...'z'. But,  
|logically, if (1..3).member?(1.5) is true, then the string
range  
|should return true as well, as 'a' <= 'aa' <= 'z'.

OK, makes sense.  Do you mean member? (and its alias
include?) should
always use discrete comparison?

							matz.

On Mar 26, 2008, at 12:31 PM, Yukihiro Matsumoto wrote:
> OK, makes sense.  Do you mean member? (and its alias
include?) should
> always use discrete comparison?

My understanding is that in the general case a range is
defined by two  
objects of compatible classes. The objects should define
succ() and <=  
or <. The members in the range are derived by applying
succ() to the  
first object, and then to the object returned by succ(),
etc, until  
the result is no longer <= or < the end of the range.

So, in the general case, all ranges are discrete.

The anomaly is ranges of floats, which you aren't allowed to
iterate  
anyway. In that case, member? becomes a comparison that the
argument  
is between the limit, because conceptually the range
(1.0..2.0)  
contains the infinite number of real values between 1 and
2.

So, yes, I think that all range membership tests should be
discrete,  
with float being the special case that you fake it out
because  
iterating over the infinite number of values would take too
long 


Dave

On Wed, Mar 26, 2008 at 7:01 PM, Dave Thomas <davepragprog.com> wrote:
>  So, in the general case, all ranges are discrete.
>
>  The anomaly is ranges of floats, which you aren't
allowed to iterate
>  anyway. In that case, member? becomes a comparison
that the argument
>  is between the limit, because conceptually the range
(1.0..2.0)
>  contains the infinite number of real values between 1
and 2.
>
>  So, yes, I think that all range membership tests
should be discrete,
>  with float being the special case that you fake it out
because
>  iterating over the infinite number of values would
take too long 

Oh, btw, not only floats but also Rationals (and maybe some
other
custom Numeric types).

And such changes would invalidate the "The Ruby
programming language"
book, section "Ranges":

" If the endpoints of the range are numbers, these
methods use the
continuous membership test, just as they did in Ruby 1.8. If
the
endpoints are not numeric, however, they instead use the
discrete
membership test. "

Thanks,
  --Vladimir

On Mar 26, 2008, at 1:15 PM, Vladimir Sizikov wrote:
>
> Oh, btw, not only floats but also Rationals (and maybe
some other
> custom Numeric types).
>
> And such changes would invalidate the "The Ruby
programming language"
> book, section "Ranges":
>
> " If the endpoints of the range are numbers, these
methods use the
> continuous membership test, just as they did in Ruby
1.8. If the
> endpoints are not numeric, however, they instead use
the discrete
> membership test. "

I'm updating that constantly as things change--that's the
joy (and  
pain) of the beta process... 

From: Dave Thomas [mailto:davepragprog.com] 
> The anomaly is ranges of floats, which you aren't
allowed to iterate  
> anyway. In that case, member? becomes a comparison that
the argument  
> is between the limit, because conceptually the range
(1.0..2.0)  
> contains the infinite number of real values between 1
and 2.

I've occasionally found it convenient to use a single Range
of Time
values, to be able to determine if a time falls in the
middle of them. A
range of Times would not, I think, be considered discrete.

I would be sad (and confused) if this behavior changed:

irb(main):006:0> t1 = Time.now
=> Wed Mar 26 13:59:15 -0600 2008
irb(main):007:0> t2 = t1 + 10
=> Wed Mar 26 13:59:25 -0600 2008
irb(main):008:0> t3 = t1 + 5.5
=> Wed Mar 26 13:59:21 -0600 2008
irb(main):009:0> (t1..t2).include?( t3 )
=> true

...despite the fact that Time#succ is defined and happens to
step
forward 1 second at a time.

I ask for validating arguments to the claim that a range is
a set of
discrete elements. I contend that it is only discrete once
it is
actually expanded/iterated. It seems more appropriate to
have
my_range.to_set.include?( foo ) when you truly want to talk
about set
membership.

Gabon:

You'd still have

t3.between? t1, t2

And you'd still have

t1..t2.cover t3


So it seems you're covered

Cheers


Dave

On Mar 26, 2008, at 3:02 PM, "Gavin Kistner"
<gavin.kistneranark.com>  
wrote:

> From: Dave Thomas [mailto:davepragprog.com]
>> The anomaly is ranges of floats, which you aren't
allowed to iterate
>> anyway. In that case, member? becomes a comparison
that the argument
>> is between the limit, because conceptually the
range (1.0..2.0)
>> contains the infinite number of real values between
1 and 2.
>
> I've occasionally found it convenient to use a single
Range of Time
> values, to be able to determine if a time falls in the
middle of  
> them. A
> range of Times would not, I think, be considered
discrete.
>
> I would be sad (and confused) if this behavior
changed:
>
> irb(main):006:0> t1 = Time.now
> => Wed Mar 26 13:59:15 -0600 2008
> irb(main):007:0> t2 = t1 + 10
> => Wed Mar 26 13:59:25 -0600 2008
> irb(main):008:0> t3 = t1 + 5.5
> => Wed Mar 26 13:59:21 -0600 2008
> irb(main):009:0> (t1..t2).include?( t3 )
> => true
>
> ...despite the fact that Time#succ is defined and
happens to step
> forward 1 second at a time.
>
> I ask for validating arguments to the claim that a
range is a set of
> discrete elements. I contend that it is only discrete
once it is
> actually expanded/iterated. It seems more appropriate
to have
> my_range.to_set.include?( foo ) when you truly want to
talk about set
> membership.
>

Dave Thomas wrote:
> 
> On Mar 26, 2008, at 1:15 PM, Vladimir Sizikov wrote:
>>
>> Oh, btw, not only floats but also Rationals (and
maybe some other
>> custom Numeric types).
>>
>> And such changes would invalidate the "The
Ruby programming language"
>> book, section "Ranges":
>>
>> " If the endpoints of the range are numbers,
these methods use the
>> continuous membership test, just as they did in
Ruby 1.8. If the
>> endpoints are not numeric, however, they instead
use the discrete
>> membership test. "
> 
> I'm updating that constantly as things change--that's
the joy (and pain) 
> of the beta process... 
> 

Hey Dave! Vladimir was quoting my book, not yours!  

Thanks for noting the book compatibility issue, Vladimir.

I have several comments on this topic:

0) I assume that Matz had well-thought-out reasons for
implementing the 
current behavior and that these are based on real-world
utility.

1) I don't think it is worth breaking compatibility with 1.8
over this 
issue.  And I think changing include? and member? would
break a 
substantial amount of code.

2) I don't believe it is correct to regard Ranges as sets of
values. 
They are something different and can have continuous or
discrete 
behavior depending on how they are used.  If you define a
range as a 
kind of set, then the behavior of include? and member? seems
wrong.  But 
  if you instead look at how those methods behave in 1.8 and
1.9 then it 
becomes clear that a range is not a set.

3) For ranges with integer endpoints, using succ is not a
reasonable way 
to implement member? and include?--it is too slow compared
to <=>. 
Therefore a discrete membership test is likely to involve
some kind of 
type checking--to ensure that the argument is of the same
class as the 
endpoints.  And I suspect that this might make some people
uncomfortable.

4) I'd suggest that include? and member? be left as they are
with their 
complex (sometimes discrete sometimes continuous) but 
backward-compatible behavior.  Ruby 1.9 already introduces a
new method 
cover? that guarantees a continuous membership test.  Let's
add yet 
another new method that guarantees a discrete membership
test.  This new 
method could be called has?.  If cover? and has? prove
satisfactory for 
Ruby 1.9 developers, then include? and member? could
possibly be 
deprecated in 2.0.

	David

On Mar 26, 2008, at 3:48 PM, David Flanagan wrote:
> 2) I don't believe it is correct to regard Ranges as
sets of values.  
> They are something different and can have continuous or
discrete  
> behavior depending on how they are used.  If you define
a range as a  
> kind of set, then the behavior of include? and member?
seems wrong.   
> But  if you instead look at how those methods behave in
1.8 and 1.9  
> then it becomes clear that a range is not a set.

David:

By definition, ranges are defined for all objects other than
a few  
built-in ones to use succ() and comparisons. Thus they are
by  
definition a representation of a set of values. The elements
in 1..3  
are the integers 1, 2, and 3. The float 1.5 is not a member
of this  
list, so member? should return false in this case. I
strongly advocate  
fixing the current behavior so that the method name and the
method  
behavior are more aligned. That way cover? can do its thing
and  
member? can do its.



Cheers


Dave

Dave Thomas wrote:
> 
> On Mar 26, 2008, at 3:48 PM, David Flanagan wrote:
>> 2) I don't believe it is correct to regard Ranges
as sets of values. 
>> They are something different and can have
continuous or discrete 
>> behavior depending on how they are used.  If you
define a range as a 
>> kind of set, then the behavior of include? and
member? seems wrong.  
>> But  if you instead look at how those methods
behave in 1.8 and 1.9 
>> then it becomes clear that a range is not a set.
> 
> David:
> 
> By definition, ranges are defined for all objects other
than a few 
> built-in ones to use succ() and comparisons. 

The "few built-in ones" include Fixnum, which is
probably the most 
commonly used case by far.  You can't treat Fixnum as an
edge case.

Thus they are by definition
> a representation of a set of values. 

A finite set of discrete values, or an infinite set of
continuous 
values?   And by whose definition?  I imagine that the
current rdoc 
comments came from the first edition of your book. But the
behavior of 
the class (since sometime in the 1.8.x series) argues that
ranges with 
numeric endpoints are not really like sets.  In Ruby, I
think that the 
MRI implementation is the ultimate "definition".

The elements in 1..3 are the
> integers 1, 2, and 3. The float 1.5 is not a member of
this list, so 
> member? should return false in this case. I strongly
advocate fixing the 
> current behavior so that the method name and the method
behavior are 
> more aligned. That way cover? can do its thing and
member? can do its.
> 

I'm not saying I agree with the current behavior.  But I
don't think 
that changing it now would be wise. Especially since it has
already 
changed once.  If that previous change was a mistake and
current 
behavior is a bug that has gone undetected for years then it
should be 
fixed in 1.8 as well as 1.9.  But I suspect that this is not
the kind of 
thing that can safely be changed in a 1.8.x point
release--it would 
break too much code.

This would have been something to fix during the lead-up to
1.9.0 when 
cover? was being introduced.  But 1.9.0 has been released,
and the Ruby 
1.9 API is, in theory, frozen. In practice, of course,
changes are still 
being made, but the bar is presumably much higher now. 
Introducing a 
new method to complement cover? and guarantee discrete
comparison would 
be one thing, but changing an existing method in an
incompatible way is 
a much bigger deal!

	David

On Mar 26, 2008, at 5:10 PM, David Flanagan wrote:
> I'm not saying I agree with the current behavior.

Then now's the time to change it, I'd say...

> This would have been something to fix during the
lead-up to 1.9.0  
> when cover? was being introduced.  But 1.9.0 has been
released, and  
> the Ruby 1.9 API is, in theory, frozen. In practice, of
course,  
> changes are still being made, but the bar is presumably
much higher  
> now.

We've added two entire classes to the built-in list since
the 1.9.0  
release, and added many, many methods to array, string, and
so on, so  
I don't think that argument really holds. We should do
what's right  
for the language _before_ 1.9 is truly frozen (if that ever
happens).  
And, as authors, we accept that our some of our descriptions
will  
gradually become incorrect (as happened to the second
edition of the  
PickAxe when the member? change was made in 1.8.6. The sky
didn't fall  
then, and the sky won't fall now.) We all agree (I think)
that member?  
is not currently doing what its name promises, and fixing it
now is a  
lot easier than fixing it later.


Cheers


Dave