Localization support

Hi Folks,

I am thinking about redesigning the i18n stuff which has
some drawbacks in
the current code. For now, it is a static class in the gui
service. I have
managed to override the methods to which I referred in a
previous mail,
and that caused the message.properties file to be parsed in
ISO-8859-1. It
is now properly read as UTF-8.

This said, I'd like to initiate a discussion about how to
fit the i18n
stuff more efficiently into SC, because we will need
internationalization
in other places than the GUI, especially with all the cool
new modules
coming in .

So, initially the i18n stuff was supposed to be moved to
util, but I'd
like to know how you feel about this. There are several
questions that
come to mind, for example, regarding translation files : one
monolithic
file, or one file for each "subset" of text ?
Shall the i18n support be a
service ? Maybe nobody cares :-p ...

Tell me what you think !

Cheers,

Jean

------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
For additional commands, e-mail: dev-helpsip-communicator.dev.java.net

Hello Jean,

lorchatsfc.wide.ad.jp wrote:
> Hi Folks,
> 
> I am thinking about redesigning the i18n stuff which
has some drawbacks in
> the current code. For now, it is a static class in the
gui service. I have
> managed to override the methods to which I referred in
a previous mail,
> and that caused the message.properties file to be
parsed in ISO-8859-1. It
> is now properly read as UTF-8.

Excellent! Could you please send your patch over so that we
could have a
look?

> This said, I'd like to initiate a discussion about how
to fit the i18n
> stuff more efficiently into SC, because we will need
internationalization
> in other places than the GUI, especially with all the
cool new modules
> coming in .
> 
> So, initially the i18n stuff was supposed to be moved
to util, but I'd
> like to know how you feel about this. There are several
questions that
> come to mind, for example, regarding translation files
: one monolithic
> file, or one file for each "subset" of text ?
Shall the i18n support be a
> service ? Maybe nobody cares :-p ...

Well, one monolithic file would be difficult to achieve
since plugins
won't be able to use it, so we might just as well try and
find a way
that would work for everyone. I am thinking of something
like defining a
"resources" directory that would be located in the
root of all bundles
that need i18n as well as accepting some convention as to
how we should
name our resource files.

Ideally a utility that would be handling i18n is going to
provide
methods which would look something like:

I18NUtility.getString(Class  myCallingClass,
                      String
"the.key.of.the.string.i.want");

When called, the method would check what is the currently
default
locale, then try and load a resource for it and if no such
resource
exists, return a value from the default one.

In terms of programming, a very easy way of implementing
I18NUtility.getString(); would be to use the
ClassLoader.getResource()
method. So in case the currently selected locale is Korean,
then we'd
have (quite roughly) the following calls:

public String getString(Class callerClass, String key)
{
    ...

    URL resourcesURL = callerClass.getClassLoader()
           
.getResource("resources/resources.txt_kr"));

    if ( resourcesURL != null)
    {
        //.. load and use the file that resourcesURL points
to
    }
    else
    {
        //asssuming resources/resources.txt is where default
resource
        //names are kept
        resourcesURL = callerClass.getClassLoader()
           
.getResource("resources/resources.txt"));
    }

    ...

    return value;
}

I don't know whether the
"callerClass.getClassLoader().getResource()"
part would work properly from inside a bundle different from
the one
that callerClass belongs to, because of all the restrictions
that OSGi
imposes on class loading. If it does then the whole thing
can be
implemented as a service. If it doesn't, we'd need to pack
it as a
lightweight lib and include it inside every bundle that uses
it.

How does this sound?

It might also be a good idea to have a bundle that would be
handling
management of the current locale - a locale manager. If we
decide to
implement the above as a service then both (locale manager
and resource
retriever) would most probably be in the same bundle. The
locale manager
would have to register a configuration GUI component in the
SC config
form and menus so that users could easily switch from one
locale to
another. It would also save the currently selected locale
to
sip-communicator.xml in case it is different from the host
system locale.

The bundle may also export a service that allows querying as
well as
persistently setting the default locale.

So what do you think about all this?

Emil

> 
> Tell me what you think !
> 
> Cheers,
> 
> Jean
> 
>
------------------------------------------------------------
---------
> To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
> For additional commands, e-mail: dev-helpsip-communicator.dev.java.net
> 
> 

------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
For additional commands, e-mail: dev-helpsip-communicator.dev.java.net

Hi Emil,

nice to hear from you 
>> It
>> is now properly read as UTF-8.
>
> Excellent! Could you please send your patch over so
that we could have a
> look?

Of course. I'll send it later today, as I am now in an
important meeting...
The patch extends ResourceBundle and Properties to support
UTF-8 as input.

As you will NOT be able to see in the patch, this used to
work pretty well
on the previous architecture, with the monolithic file and
the static
class. But you described the problems with a single file
better in your
previous mail.

> Well, one monolithic file would be difficult to achieve
since plugins
> won't be able to use it, so we might just as well try
and find a way
> that would work for everyone. I am thinking of
something like defining a
> "resources" directory that would be located
in the root of all bundles
> that need i18n as well as accepting some convention as
to how we should
> name our resource files.

This suits me fine. It means that when preparing a new
bundle, everyone
should think about putting one file (at least !) in the
resources
directory. Am I right ? And does it sound feasible ?

> Ideally a utility that would be handling i18n is going
to provide
> methods which would look something like:
>
> I18NUtility.getString(Class  myCallingClass,
>                       String
"the.key.of.the.string.i.want");
> When called, the method would check what is the
currently default
> locale, then try and load a resource for it and if no
such resource
> exists, return a value from the default one.

This is almost what we have. In fact, we are based on the
ResourceBundle
class, and use the getString method of that class. However,
you have to do
some kind of initialization first. This is where we have a
problem because
the path is all messed up and I do not understand how this
works in Java
:-P.

> In terms of programming, a very easy way of
implementing
> I18NUtility.getString(); would be to use the
ClassLoader.getResource()
> method. So in case the currently selected locale is
Korean, then we'd
> have (quite roughly) the following calls:
>
> public String getString(Class callerClass, String key)
> {
>     ...
>
>     URL resourcesURL = callerClass.getClassLoader()
>            
.getResource("resources/resources.txt_kr"));
>
>     if ( resourcesURL != null)
>     {
>         //.. load and use the file that resourcesURL
points to
>     }
>     else
>     {
>         //asssuming resources/resources.txt is where
default resource
>         //names are kept
>         resourcesURL = callerClass.getClassLoader()
>            
.getResource("resources/resources.txt"));
>     }
>
>     ...
>
>     return value;
> }
>
> I don't know whether the
"callerClass.getClassLoader().getResource()"
> part would work properly from inside a bundle different
from the one
> that callerClass belongs to, because of all the
restrictions that OSGi
> imposes on class loading. If it does then the whole
thing can be
> implemented as a service. If it doesn't, we'd need to
pack it as a
> lightweight lib and include it inside every bundle that
uses it.

Well, I had a hell of a headache with paths already, so I am
not afraid to
do more testing. What I am afraid of, is whether or not we
can have access
to a resources/ directory at the root of all bundles ? And
does it sound a
bad idea to use resources/ directory in the bundle directory
itself
instead ? It is not a very centralized process though.

Then, about your example, I have a very big concern. I
wanted to use
ResourceBundle class because it parses a file and uses hash
tables to
store value pairs. This is very efficient, especially when
using a lot of
text. Indeed, I do not want to open the file and search for
a string each
time. So if we end up using something else than
ResourceBundle, we have to
cache it. In addition, ResourceBundle is neat because it is
hierarchical,
and can return a resource from another "upper"
language in the hierarchy,
if the string is not found in a specific "lower"
language. Disclaimer : I
do not intend to classify languages, but usually, at least
english has all
the localized strings. And for example, canadian french can
resort to
french, or the other way round.
>
> How does this sound?

Sounds good for me. I'll try to figure out how the path
nightmare sorts
out using ClassLoader  stuff.

> It might also be a good idea to have a bundle that
would be handling
> management of the current locale - a locale manager. If
we decide to
> implement the above as a service then both (locale
manager and resource
> retriever) would most probably be in the same bundle.
The locale manager
> would have to register a configuration GUI component in
the SC config
> form and menus so that users could easily switch from
one locale to
> another. It would also save the currently selected
locale to
> sip-communicator.xml in case it is different from the
host system locale.

I planned to get the locale from the xml config file, but
your description
of the process is much better than I could have dreamt
of...

> The bundle may also export a service that allows
querying as well as
> persistently setting the default locale.
>
> So what do you think about all this?

Well, I now have some pointers to push the effort a bit
further. I'll tell
you if I meet any dead-end.

By the way, if someone else has insights to share about this
problem, do
feel free to tell !

Cheers,

Jean

------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
For additional commands, e-mail: dev-helpsip-communicator.dev.java.net

Hello Jean,

Nice to hear from you too!

lorchatsfc.wide.ad.jp wrote:
> Hi Emil,
> 
> nice to hear from you 
>>> It
>>> is now properly read as UTF-8.
>> Excellent! Could you please send your patch over so
that we could have a
>> look?
> 
> Of course. I'll send it later today, as I am now in an
important meeting...
> The patch extends ResourceBundle and Properties to
support UTF-8 as input.

Great.

> As you will NOT be able to see in the patch, this used
to work pretty well
> on the previous architecture, with the monolithic file
and the static
> class. But you described the problems with a single
file better in your
> previous mail.

Okay, we'll talk about this once we see the patch but I'm
relatively
confident it should also work with the separate files.

>> Well, one monolithic file would be difficult to
achieve since plugins
>> won't be able to use it, so we might just as well
try and find a way
>> that would work for everyone. I am thinking of
something like defining a
>> "resources" directory that would be
located in the root of all bundles
>> that need i18n as well as accepting some convention
as to how we should
>> name our resource files.
> 
> This suits me fine. It means that when preparing a new
bundle, everyone
> should think about putting one file (at least !) in the
resources
> directory. Am I right ? And does it sound feasible ?

If they want to use our utility then they should be prepared
to have at
least one resource file. If they want to use our utility
then they'd
also have to make sure that when their bundle is created,
their resource
files are jar-ed inside a "resources" directory
directly under the root
of the bundle.

>> Ideally a utility that would be handling i18n is
going to provide
>> methods which would look something like:
>>
>> I18NUtility.getString(Class  myCallingClass,
>>                       String
"the.key.of.the.string.i.want");
>> When called, the method would check what is the
currently default
>> locale, then try and load a resource for it and if
no such resource
>> exists, return a value from the default one.
> 
> This is almost what we have. In fact, we are based on
the ResourceBundle
> class, and use the getString method of that class.
However, you have to do
> some kind of initialization first. This is where we
have a problem because
> the path is all messed up and I do not understand how
this works in Java
> :-P.

Can you be more specific? What do you need to do in the
initialization
phase? Would it be ok if we did it at bundle startup (i.e.
inside the
Activator.start() method)?

> <snip> 
> ...
> </snip> 
> Well, I had a hell of a headache with paths already, so
I am not afraid to
> do more testing. What I am afraid of, is whether or not
we can have access
> to a resources/ directory at the root of all bundles ?


I think that we can. As long as we get our hands on the
right class
loader, we can go check the content of directories stored in
its source
bundle. I'll do some testing and let you know.

> And does it sound a
> bad idea to use resources/ directory in the bundle
directory itself
> instead ? It is not a very centralized process though.

That would be more difficult to have since it would be the
installer's
responsibility to create and fill the directory which means
that it
won't work for plugins installed later.

> Then, about your example, I have a very big concern. I
wanted to use
> ResourceBundle class because it parses a file and uses
hash tables to
> store value pairs. This is very efficient, especially
when using a lot of
> text. Indeed, I do not want to open the file and search
for a string each
> time. 

True ... I hadn't really thought of that but we can easily
make our
ResourceUtility service cache resource files against the
ClassLoaders
that were used for testing them.

> So if we end up using something else than
ResourceBundle, we have to
> cache it. In addition, ResourceBundle is neat because
it is hierarchical,
> and can return a resource from another
"upper" language in the hierarchy,
> if the string is not found in a specific
"lower" language. Disclaimer : I
> do not intend to classify languages, but usually, at
least english has all
> the localized strings. And for example, canadian french
can resort to
> french, or the other way round.

I agree that it would indeed be great to stick with
ResourceBundles, and
I'd have to confess, that I don't really see why we
couldn't. Is there
something I am missing?

>> How does this sound?
> 
> Sounds good for me. I'll try to figure out how the path
nightmare sorts
> out using ClassLoader  stuff.
> 
>> It might also be a good idea to have a bundle that
would be handling
>> management of the current locale - a locale
manager. If we decide to
>> implement the above as a service then both (locale
manager and resource
>> retriever) would most probably be in the same
bundle. The locale manager
>> would have to register a configuration GUI
component in the SC config
>> form and menus so that users could easily switch
from one locale to
>> another. It would also save the currently selected
locale to
>> sip-communicator.xml in case it is different from
the host system locale.
> 
> I planned to get the locale from the xml config file,
but your description
> of the process is much better than I could have dreamt
of...

Ha, come on!

I'll go for that resource testing now and let you know of my
findings.

Cheers
Emil

>> The bundle may also export a service that allows
querying as well as
>> persistently setting the default locale.
>>
>> So what do you think about all this?
> 
> Well, I now have some pointers to push the effort a bit
further. I'll tell
> you if I meet any dead-end.
> 
> By the way, if someone else has insights to share about
this problem, do
> feel free to tell !
> 
> Cheers,
> 
> Jean
> 
>
------------------------------------------------------------
---------
> To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
> For additional commands, e-mail: dev-helpsip-communicator.dev.java.net
> 
> 

------------------------------------------------------------
---------
To unsubscribe, e-mail: dev-unsubscribesip-communicator.dev.java.net
For additional commands, e-mail: dev-helpsip-communicator.dev.java.net