Dragging and dropping with orca

Hello,

Can anyone on this list tell me if there is a way to use
Orca to  
emulate the drag-and-drop mouse function? I currently use
VoiceOver  
with Tiger (the screen reader provided with Mac OS 10),
which does  
not allow blind computer users to drag and drop. This is a
serious  
inconvenience that I am hoping I can circumvent by using
Orca and  
Ubunto.

Cordially,
Rafael Bejarano
_______________________________________________
Orca-list mailing list
Orca-listgnome.org
http
://mail.gnome.org/mailman/listinfo/orca-list
Visit http://live.gnome.org/Orca
 for more information on Orca

On 27 May 2007 at 2:47, Rafael Bejarano wrote:

> Can anyone on this list tell me if there is a way to
use Orca to 
> emulate the drag-and-drop mouse function? I currently
use VoiceOver  


Unfortunately not at this time, but i do hope this feature
would be implemented in some way.
Should i file some sort of bug to get this thought over?
-- 
/Krister
_______________________________________________
Orca-list mailing list
Orca-listgnome.org
http
://mail.gnome.org/mailman/listinfo/orca-list
Visit http://live.gnome.org/Orca
 for more information on Orca

Hi,

Gerd Kohlberger (CCed) is working on some mouse enhancements
as a summer 
of code project, which will include simulating clicks and
click+hold. If 
we can get Orca to send some commands to the mouse
application then drag 
and drop should be possible IMO. You would navigate to the
icon you 
wanted on the desktop or in Nautilus, issue a command to
'grab' it and 
then change to another window and issue the release command.
Spec: 
https://wiki.ubuntu.com/Accessibility/Specs/MouseTweaks

How will this differ from cutting (Ctrl+X) and pasting
(Ctrl+V) though? 
What could you do with drag and drop that you cannot do with
cut and paste?

Henrik

Rafael Bejarano wrote:
> Hello,
>
> Can anyone on this list tell me if there is a way to
use Orca to  
> emulate the drag-and-drop mouse function? I currently
use VoiceOver  
> with Tiger (the screen reader provided with Mac OS 10),
which does  
> not allow blind computer users to drag and drop. This
is a serious  
> inconvenience that I am hoping I can circumvent by
using Orca and  
> Ubunto.
>
> Cordially,
> Rafael Bejarano
> _______________________________________________
> Orca-list mailing list
> Orca-listgnome.org
> http
://mail.gnome.org/mailman/listinfo/orca-list
> Visit http://live.gnome.org/Orca
 for more information on Orca
>
>   

_______________________________________________
Orca-list mailing list
Orca-listgnome.org
http
://mail.gnome.org/mailman/listinfo/orca-list
Visit http://live.gnome.org/Orca
 for more information on Orca

Here are some of my thoughts on this.

In most cases where dragging and dropping is needed (because
of no
keyboard commands for the control) it normally comes down to
a poor
implementation, particularly when it comes to accessibility.
In these
cases its normally imaginable that there could be keyboard
equivilents
to do what the dragging and dropping is doing whilst
allowing those who
don't want to use a keyboard to click to their hearts
content. As an
example, in sound editing software, when the wave view
control gets
focus you could have it that you can move the initial marker
(the front
of the highlight, with left and right, and move the right
edge of the
highlight with shift and cursor left and right. In this
example, what
should be spoken and brailled for such a control, well it
could give
current position of moved marker (left or right edge of
highlight) and
whether it is sitting on any other marker such as a track
marker. You
may imagine where I am now heading, yes you could use cut,
copy and
paste for the highlight where you may normally drag and drop
that
selection.

I can think of a few cases where possibly keyboard shortcuts
may be
difficult to implement, although may be possible, but these
uses tend to
be very visual in the operation that is being carried out
and would be
extremely, if not impossible, to represent via braille or
speech due to
the highly graphical nature of the material which is being
operated on
(eg. in image manipulation, could you make a drawing package
fully
accessible even if the keyboard shortcuts were there? What
should the
drawing area speak to the user?).

One thing with linux, is that a lot of the functionality is
provided
from an underlying library, and normally there are various
front ends
for it (eg. GTK or QT interfaces), and as the text console
is still
quite used in linux, sometimes if the graphical interface is
inaccessible you may just have to roll up your sleeves and
get your
hands dirty and use the text console. May be in suggesting
this I am
foolish on such a list, but I think the text console is
normally much
more accessible, and only use gnome where text based systems
fail (eg.
those awful websites that near enough require you to use a
browser they
want you to use and firefox ends up being the only
accessible Linux
solution for them).

From
Michael Whapples
On Wed, 2007-05-30 at 16:56 +0300, Veli-Pekka Tätilä
wrote:
> kristerkristersplace.ws wrote:
> > On 29 May 2007 at 22:39, Henrik Nilsen Omma wrote:
> >> How will this differ from cutting (Ctrl+X) and
pasting (Ctrl+V)
> >> though? What could you do with drag and drop
that you cannot do with
> >> cut and paste?
> > There are situations, such as in certain audio
editing and music
> > software wher you are to manipulate widgets and
such and in that
> > case, you can't copy/
> > paste.
> Exactly, and not just audio apps iether. Think of
diagram editing or 
> anything remotely graphical that's based on direct
manipulation. Any 
> cursorless area in which the user has to use the mouse
to select stuff 
> presents similar problems. many readers implement
D&D, that's drag and drop 
> not dungeons and dragons <ssmile>, as marking the
source object for dragging 
> and the destination object for dropping. But another
equally useful thing 
> would be the ability to drag with the left or right
button an arbitrary 
> vertical, horizontal or circular amount measured in
pixels. This kind of 
> dragging is used to manipulate controls like knobs,
which is not about 
> dragging one object to another.
> 
> Yet another thing, in which at least the MAc requires
drag and drop, is 
> lists in which you have to re-arrange items by
preference e.g. preferred 
> languages on the Web. In addition to screen reader
based D&D, I've seen 
> up/down buttons and keyboard interfaces for moving e.g.
alt+arrows.
> 
> And howabout re-arranging and re-sizing columns in what
people call tables, 
> multi-column list views or deetails views depending on
context? Windows does 
> not provide a keyboard interface for any of this in the
control directly, 
> and if the Linux folks have been borrowing without
thinking, I suppose 
> neither does Gnome. I honestly think MS made a mistake
with that particular 
> control and having to suffer from it ten years in
various apps, it is 
> extremely annoying to see the same poor keyboard
interface being duplicated, 
> say in OS X. Sory, I just feel strongly about this, and
wanted to get the 
> point across (ho offense ment, <smile>). I
honestly hope Gnome and all the 
> rest of the Linux GUis do better keybordwise. So, the
question, can you 
> re-arrange columns in a multi-column list from the
keyboard in Gnome? In 
> WIndows, screne reader based D&D or duplicating the
functionality in a 
> dialog box (think OE or Explorer's choose columns) are
some of the 
> work-arounds.
> 
> Back to mousy things, though, the fundamental problem
is at the app end, 
> though, failing to provide a keyboard interface for
custom controls like 
> knobs, sliders, points in graphs etc...  I see no
reason why knobs could not 
> have the same keyboard interface as sliders do:
up/right for small 
> increment, pg up/down for larger units and home/end for
 reaching min and 
> max. Its just that once things get custom enough,
people stop payihng 
> attention to keyboard interfaces, which is sad. And its
quite common that 
> you even cannot type in or see values directly, because
the original thing 
> they are emulating or modelling could not do that
either, which annoys the 
> heck out of me. It happens an awful lot in the synth
plug world for no real 
> reason.
> 
> How is the Linux music software in this regard? Most
Vst plugs and hosts are 
> keyboard nightmares to say the least, and I'm afraid
this mousy philosophy 
> might have carried over to LInux as wel. How is ardour,
for example, which 
> should be GTK+ 2 based and thus potentially accessible?
a friend of mine has 
> been hyping that one a lot lately. He also runs VST
plugs under WIne but no 
> ORca there, of course.
> 
> > want to put an audio loop on a track you have to
drag it there.
> Not only that but you have to drag, too, when you want
to extend a pattern. 
> from a keyboard user point of view, this makes no sense
at all. I mean, if I 
> know I want 10 times pattern x, 12 times pattern y and
10 times pattern z, 
> that's a lot of D&D which can be very slow
full-sscreen magnified. It would 
> be an order of magnitude faster for me to simply supply
a repeat count.
> 
> -- 
> With kind regards Veli-Pekka Ttil (vtatilamail.student.oulu.fi)
> Accessibility, game music, synthesizers and
programming:
> http://www.stude
nt.oulu.fi/~vtatila/ 
> 
> 
> 
_______________________________________________
Orca-list mailing list
Orca-listgnome.org
http
://mail.gnome.org/mailman/listinfo/orca-list
Visit http://live.gnome.org/Orca
 for more information on Orca

Hi Michael,
You have some very good points in there.

Yes, the fact that most keyboard inaccessible controls that
require D&D 
could simply have a keyboard interface is what I was after,
too. My point is 
that, even in the LInux world, such accessibility blundres
don't seem to get 
fixed instantly at all, so it would be still highly useful
to have the D&D 
functionality in Orca, too, as a reader specific
work-around.

But could not the accessibility API support things like that
screen reader 
independently? At least in the MS world the same lib is used
for 
accessibility and GUI testing, meaning that you can also
interact with 
controls programmatically.

About sound editing, actually Sound FOrge here in the Wintel
world has much 
the keyboard interface and screen reader support you go onto
describe. You 
do have to switch the selection between the two ends but
other than that its 
great. I'm still missing the ability to select 10 % of the
viewport by page 
up/down in Audacity. It works in Sound Forge, after all.

You're right that many apps that are truely based on direct
manipulation and 
graphics would not really benefit from a keybord interface.
But even some 
things that are conventionally thought of as graphical could
be, if you'd 
like to rethink, made accessible. One such thing is a graph
with draggable 
points. There's no reason why you could not manage the
points in a list 
view, for example, and mentally interpolate as needed.

Lastly, this is geting a bit odd but true Linux still is a
command-line OS 
and no matter how many graphical front-ends will be written,
it will be a 
command-line OS. Thus it pays off to learn that
command-line. Which reminds 
me, how well does it work in Orca?

But in that context I've found that I'm not a command-line
guy in general. 
Apparently it is quite possible to enjoy programming, text
processing and 
scripting (think, Perl, Ruby and Regexp) while still wanting
to do things 
graphically, when wearing your user hat. I'm like that. Its
a testament to 
the ease of use and relative power of a GUi system that even
some people who 
handle it as text from the keybord find it more enjoyable
than the 
command-line.

I'm sure I'm in the minority but that's how things stand.
let's take a 
concrete example like file navigation. WHen I do it
graphically I can 
navigate via type ahead and get instant spoken feedback of
the currently 
selected item. WHen you add the ability to type in multiple
letters, wild 
cards and searching by extension plus sorting, its very
efficient. Where as 
when i use ls and cd to get around it just feels clumsy,
much more error 
prone and a lot slower, too, even though I'm a touch typist.
My sighted 
friends tell me multi-item file auto-complete and the smart
guessing that 
ZSH do help immensly but both are much quicker to scan
visually than they 
are using speech.

But Hey, I'm getting OT, so if this gets much more OT, let's
move it 
off-list, right?

-- 
With kind regards Veli-Pekka Tätilä (vtatilamail.student.oulu.fi)
Accessibility, game music, synthesizers and programming:
http://www.stude
nt.oulu.fi/~vtatila/

Michael Whapples wrote:
> Here are some of my thoughts on this.
>
> In most cases where dragging and dropping is needed
(because of no
> keyboard commands for the control) it normally comes
down to a poor
> implementation, particularly when it comes to
accessibility. In these
> cases its normally imaginable that there could be
keyboard equivilents
> to do what the dragging and dropping is doing whilst
allowing those
> who don't want to use a keyboard to click to their
hearts content. As
> an example, in sound editing software, when the wave
view control gets
> focus you could have it that you can move the initial
marker (the
> front of the highlight, with left and right, and move
the right edge
> of the highlight with shift and cursor left and right.
In this
> example, what should be spoken and brailled for such a
control, well
> it could give current position of moved marker (left or
right edge of
> highlight) and whether it is sitting on any other
marker such as a
> track marker. You may imagine where I am now heading,
yes you could
> use cut, copy and paste for the highlight where you may
normally drag
> and drop that selection.
>
> I can think of a few cases where possibly keyboard
shortcuts may be
> difficult to implement, although may be possible, but
these uses tend
> to be very visual in the operation that is being
carried out and
> would be extremely, if not impossible, to represent via
braille or
> speech due to the highly graphical nature of the
material which is
> being operated on (eg. in image manipulation, could you
make a
> drawing package fully accessible even if the keyboard
shortcuts were
> there? What should the drawing area speak to the
user?).
>
> One thing with linux, is that a lot of the
functionality is provided
> from an underlying library, and normally there are
various front ends
> for it (eg. GTK or QT interfaces), and as the text
console is still
> quite used in linux, sometimes if the graphical
interface is
> inaccessible you may just have to roll up your sleeves
and get your
> hands dirty and use the text console. May be in
suggesting this I am
> foolish on such a list, but I think the text console is
normally much
> more accessible, and only use gnome where text based
systems fail (eg.
> those awful websites that near enough require you to
use a browser
> they want you to use and firefox ends up being the only
accessible
> Linux solution for them). 

_______________________________________________
Orca-list mailing list
Orca-listgnome.org
http
://mail.gnome.org/mailman/listinfo/orca-list
Visit http://live.gnome.org/Orca
 for more information on Orca

As some of this is slightly of topic, I will try and keep it
short, but
I think while the idea of accessibility in general is not
the purpose of
this list, it certainly may bring up some useful points
relating to
Orca.

When using the command line, I normally don't use orca in
the
gnome-terminal, I switch to a proper text console and use
speakup for
that. I just feel with all the other features of orca for
working in a
GUI, it is simpler to work in a proper text console with
something
really designed for the command line. While my view may
differ on amount
of command line usage to some, I may not quite go as far as
some (look
at edbrowse, www.eklhad.net/linux/app), I prefer the 2D
layout of
something like elinks.

Regarding the visual cases where keyboard shortcuts don't
have value for
screen readers, I was more thinking of photo editing, or to
use a
windows example paint. Graph accessibility (data graphs)
would be
something I would like to see happen.

As for whether orca should have a drag and drop function as
the
situation as it could be doesn't exist and dragging and
dropping may be
the only option, I have mixed views. If orca does have the
feature,
developers don't feel that keyboard shortcuts will add to
the
accessibility of their app as orca allows us to drag and
drop, but if
orca doesn't have it then we are stuck in the mean time
until (if ever)
such a situation happens. Now yes I know as a user, the
first isn't
really satisfactory, keyboard shortcuts would be better than
using an
orca drag and drop feature, as in some of these cases if the
keyboard
navigation doesn't exist then the information for orca may
not exist.

Also your comment about the accessibility API, yes all
communication
from the app should be transported to orca through the
accessibility
layer, we don't want to have apps tied to one screen reader,
what about
LSR, or even if something better than both of them comes
along. Also
accessibility APIs could be used for app testing, which
would test the
API even more.

From
Michael Whapples
On Wed, 2007-05-30 at 18:22 +0300, Veli-Pekka Tätilä
wrote:
> Hi Michael,
> You have some very good points in there.
> 
> Yes, the fact that most keyboard inaccessible controls
that require D&D 
> could simply have a keyboard interface is what I was
after, too. My point is 
> that, even in the LInux world, such accessibility
blundres don't seem to get 
> fixed instantly at all, so it would be still highly
useful to have the D&D 
> functionality in Orca, too, as a reader specific
work-around.
> 
> But could not the accessibility API support things like
that screen reader 
> independently? At least in the MS world the same lib is
used for 
> accessibility and GUI testing, meaning that you can
also interact with 
> controls programmatically.
> 
> About sound editing, actually Sound FOrge here in the
Wintel world has much 
> the keyboard interface and screen reader support you go
onto describe. You 
> do have to switch the selection between the two ends
but other than that its 
> great. I'm still missing the ability to select 10 % of
the viewport by page 
> up/down in Audacity. It works in Sound Forge, after
all.
> 
> You're right that many apps that are truely based on
direct manipulation and 
> graphics would not really benefit from a keybord
interface. But even some 
> things that are conventionally thought of as graphical
could be, if you'd 
> like to rethink, made accessible. One such thing is a
graph with draggable 
> points. There's no reason why you could not manage the
points in a list 
> view, for example, and mentally interpolate as needed.
> 
> Lastly, this is geting a bit odd but true Linux still
is a command-line OS 
> and no matter how many graphical front-ends will be
written, it will be a 
> command-line OS. Thus it pays off to learn that
command-line. Which reminds 
> me, how well does it work in Orca?
> 
> But in that context I've found that I'm not a
command-line guy in general. 
> Apparently it is quite possible to enjoy programming,
text processing and 
> scripting (think, Perl, Ruby and Regexp) while still
wanting to do things 
> graphically, when wearing your user hat. I'm like that.
Its a testament to 
> the ease of use and relative power of a GUi system that
even some people who 
> handle it as text from the keybord find it more
enjoyable than the 
> command-line.
> 
> I'm sure I'm in the minority but that's how things
stand. let's take a 
> concrete example like file navigation. WHen I do it
graphically I can 
> navigate via type ahead and get instant spoken feedback
of the currently 
> selected item. WHen you add the ability to type in
multiple letters, wild 
> cards and searching by extension plus sorting, its very
efficient. Where as 
> when i use ls and cd to get around it just feels
clumsy, much more error 
> prone and a lot slower, too, even though I'm a touch
typist. My sighted 
> friends tell me multi-item file auto-complete and the
smart guessing that 
> ZSH do help immensly but both are much quicker to scan
visually than they 
> are using speech.
> 
> But Hey, I'm getting OT, so if this gets much more OT,
let's move it 
> off-list, right?
> 
> -- 
> With kind regards Veli-Pekka Tätilä (vtatilamail.student.oulu.fi)
> Accessibility, game music, synthesizers and
programming:
> http://www.stude
nt.oulu.fi/~vtatila/
> 

_______________________________________________
Orca-list mailing list
Orca-listgnome.org
http
://mail.gnome.org/mailman/listinfo/orca-list
Visit http://live.gnome.org/Orca
 for more information on Orca