new i18n-system

niko · 2005-04-04 05:20:14

i submitted a patch for a new i18n-system:
http://xinha.python-hosting.com/ticket/98

one question i would like to clear bevore submitting the patch:

gogo wrote:

niko wrote:
what happens with different charsets? If a language-file uses utf-8 (as
most does) but xinha runs as iso-8859-1 the special cahrs can't be displayed.
In so far as character sets, I guess we will need to pass back to the backend/whatever the character set required.

yes, that would be the easiest solution,
BUT: the HTMLArea._lc doesn't have access to the editor-object, and so can't read out the current charset!
...an one page we can only use one charset....
currently we define the chraset in HTMLArea.Config:
this.charSet = HTMLArea.is_gecko ? document.characterSet : document.charset;

....do we really need it here? if not we could use HTMLArea.is_gecko ? document.characterSet : document.charset in the _lc-function too! (as it can be changed through the config-object)

niko · 2005-04-04 10:01:48

i did some investigation on Charset-Convertion.
one possibility is to use php.net/iconv - but this might not be avaliable on every system

the second possibility would be the ConvertCharset-class from here: http://mikolajj.republika.pl/

so, what do you think?

gogo · 2005-04-05 06:48:30

BUT: the HTMLArea._lc doesn't have access to the editor-object, and so can't read out the current charset!

All javascript strings are internally representad as unicode data anyway (in the javascript engine), the encoding (utf-8) is only for getting strings "into" javascript strings, and "out of" them if necessary. Once inside the string variable (object) they are just plain old Unicode without any encoding.

I think that we should use UTF-8 only for language files, translators can convert to/from thier "preferred" characterset to utf-8 if they wish, but the language files themselves should be utf-8.

XMLHttpRequest (used by _geturlcontent) is supposed to determine from the character-encoding heading set by the server on loading the language (file) that the data it's loading is utf-8 (provided the server says that it is) and make that a unicode javascript string (object) corectl.

Infact, MSXML apparently ignores the characterset returned by the server and instead ALWAYS decodes the text (in "responseText") as utf-8 (http://www.forum4designers.com/message197591.html) , so infact, using _geturlcontent() simply isn't going to work with any other charactersets anway (in IE). The only way to stop that is to use either a BOM (Byte Order Mark) indicating it's a different UNICODE encoding (ie you can't use a BOM to indicate it's ISO-8859-1, that's not unicode), or to wrap it in XML which specifies the characterset and use responseXML to get at the data.

I think the answer is simple, have all language data mandatorally utf-8, if translators write in another characterset we can convert that before adding it to xinha (using iconv etc). For IE it would probably work just fine (because it always assumes utf-8 unless a BOM is present indicating one of the other unicode encodings (utf-16 etc)), for Mozilla, I think the implementor just has to make sure that thier server sends correct character encoding of utf-8 for these files - unless we can find a way to force the Gecko XMLHTTPRequest to interpret them as utf-8 nomatter what the server calls them, which would be preferable.

Infact, this extends to all javascript files etc that is loaded as part of Xinha, all files should be utf-8 (although I expect most files apart from the language data wouldn't need to use any double byte characters anyway so it's a largely moot point).

niko · 2005-04-05 09:26:12

Thanks for your investigation on that!

i didn't expect it being that easy

i just commited the new i18n-system
http://xinha.python-hosting.com/changeset/60

please review my changes!

mokhet · 2005-04-05 10:39:13

hi, just update my local version to svn version and i have two little things

humm, where are the files en.js now ? cant find one anywhere, all have been removed in #60, there must be somewere, but where, i cant say

looking at /lang/fr.js, i find this code (i cut it a bit, cause it would not be readable)

  "You are in TEXT MODE.  Use the [<>] button to switch back to WYSIWYG.": "Vous êtes en ...",
  "The full ...",

It's supposed to be

  "You are in TEXT MODE.  Use the [<>] button to switch back to WYSIWYG.": "Vous êtes en ..." +
  "The full ...",

I just checked the file /lang/de.js and it gots the same problem.

The problem is located in the ending char which needs to be a + and not a ,
This text is for a dialog box and is kinda long, it has been truncated on multiple lines.

There's also some utf-8 problems in /lang/es.js dont think it's police problem as the one i have in he.js or ja.js Because i use spanish documents with not encoding issue usually, so it's probably a problem on this file.

guillaumed · 2005-04-05 12:27:56

Niko,

As words and sentences don't have the same length in all languages, the position and witdh and height of dialogs some be also defined in your module...

niko · 2005-04-06 04:57:06

@mokhet:
no, the "," is correct!
although there is a problem with the full-screen-text - but that does not really matter as it will be nowhere used and could be removed.

There's also some utf-8 problems in /lang/es.js dont think it's police problem as the one i have in he.js or ja.js Because i use spanish documents with not encoding issue usually, so it's probably a problem on this file.

yes, something went wrong there... but i don't know how to correct it... probably the easiest would be if sombebody posts a corrected version

As words and sentences don't have the same length in all languages, the position and witdh and height of dialogs some be also defined in your module...

wouldn't it be easier to dynamically determine the size of the content in the dialog and resize it according to that?
(some dialogs work allready like that i think)

guillaumed · 2005-04-06 06:23:55

wouldn't it be easier to dynamically determine the size of the content in the dialog and resize it according to that?

Right for alerts, wrong for popup dialogs...

mokhet · 2005-04-06 06:58:01

niko wrote:

no, the "," is correct!

Damn, i knew my javascript knowledge were poor but not as much.

var mytab = {
 "keyA": "valueA",
 "keyB": "valueB1", "valueB2", "valueB3",
 "keyC": "valueC"
};

alert(mytab["keyB"]);

==> missing : after property id

this kind of thing is valid in the actual context ??? That's doesnt make any sense for me, looks totally invalid. damn.
Well, you know better than me so oki, but it perturbs me A LOT.

mokhet · 2005-04-06 07:23:04

after a lot more of investigation, you are of course right but in fact, it is missing some texts and some of the backslashes, i'll send you an updated one when i'll have understood how the new system is working.

are you sure _editor_lang = "fr"; is used at all ? Even with this line, my texts (correctly translated in revision 58) are not translated at all anymore in revision 60 and remains in english.

niko · 2005-04-06 08:25:23

yes, _editor_lang is used!
and its working for me (revision 60) - the old language files won't work anymore - the syntax has changed a bit...

guillaumed · 2005-04-06 09:11:39

Niko,

Please I need some help to understand your approach.. I used the charset Latin1 in my page, lang/ss.js is in utf8.. I expected to get a lot of hieroglyph on my screen but not... I'm curious to know what have you done...

Last edited by guillaumed (2005-04-06 11:54:31)

niko · 2005-04-07 02:42:34

using iso-8859-1 and de.js (is in utf-8) it worked for me!

read gogo's post!

....you might have to configure your web-server that he tells firefox that these are utf-8 encodes files.
(my webserver just sends Content-Type: application/x-javascript and it works)

Last edited by niko (2005-04-07 02:44:13)

guillaumed · 2005-04-08 02:27:11

Hi Niko,

I wonder if this ticket is related to the new i18n module ... http://xinha.python-hosting.com/ticket/125

Any chance?

niko · 2005-04-08 02:58:01

no, i can't imagine

    if (HTMLArea.is_gecko && this._doc.designMode != 'on')
    {
      try{HTMLArea.last_on.designMode = 'off';} catch(e) { }
      if(this._iframe.style.display == 'none')
      {
        this._iframe.style.display = '';
        this._doc.designMode = 'on';
        this._iframe.style.display = 'none';
      }
      else
      {
        this._doc.designMode = 'on';
      }
    }
    else
    {
      this._doc.body.contentEditable = true;
    }

this "only" sets the editor into design-mode.

guillaumed · 2005-04-08 03:50:42

Yes but there is a side effect somewhere that I hadn't before which raise the problem there and also in simple text command (I get a DOM_BAD_URI error)
This problem arise when the page is redrawn and deseapear if a reload the source...

niko · 2005-04-09 03:28:16

with revision 59 it did work?
very strang... can you reproduce this?

guillaumed · 2005-04-09 03:58:04

Yes... All the time

Trying to find the source, for the moment I can track than FF refuse to set the designMode "on" on the doc (why???) and then everything in going wrong after...

It seems that this "feature" was seen before but not really fixed..

htmlarea.js :

HTMLArea.prototype.activateEditor = function()
{
if (HTMLArea.is_gecko && this._doc.designMode != 'on')
{
try{HTMLArea.last_on.designMode = 'off';} catch(e) { } // <---

doc = editor._iframe.contentWindow.document;
if (!doc) {
// Try again..
// FIXME: don't know what else to do here. Normally
// we'll never reach this point.
if (HTMLArea.is_gecko) {
setTimeout(function() { editor.initIframe()}, 50); <-- I don't like that!!!
return false;

HTMLArea.prototype.focusEditor = function() {
switch (this._editMode) {
// notice the try { ... } catch block to avoid some rare exceptions in FireFox
// (perhaps also in other Gecko browsers). Manual focus by user is required in
// case of an error. Somebody has an idea?
case "wysiwyg" :
try
{
// We don't want to focus the field unless at least one field has been activated.
if(HTMLArea.last_on)
{
this.activateEditor();
this._iframe.contentWindow.focus();
}

I not sure but I think it is also related to the content randomly diseappear problem.. see ticket #93 and perhaps #129. In my page I can see the problem first because the focus is lost when the page are changed (the focus is not set on the new textarea created), then I can retrieve the focus on reloading source..

Any idea???

Last edited by guillaumed (2005-04-09 04:03:16)

mokhet · 2005-04-09 07:19:41

niko wrote:

yes, _editor_lang is used!

Hello everyone, finally managed to get all this working. The problem i had was quite simple, the translation system is working only with online documents. When testing in local, the texts are always in english.

local urls i have tested (and not working, but it's only local so i dont think it's required to fix by now, but it had make me ponder a lot

_editor_url  = '../';

_editor_url  = 'file:///c:/Documents%20and%20Settings/mokhet/Bureau/SVN/xinha/';

_editor_url = '/home/laurent/cvs/makeai/js/xinha/';

none of them worked in local, but worked PERFECT at the moment i dropped the test file online.

Last edited by mokhet (2005-04-11 16:56:37)

guillaumed · 2005-04-09 10:10:00

Find the problem....

Seems to be a bug in Mozilla, if you write into a doc before setting designMode="On" then pb...

Ticket raised..

Sorry Niko for having suspected your excellent job...

Last edited by guillaumed (2005-04-09 10:11:30)

niko · 2005-04-11 03:14:05

Sorry Niko for having suspected your excellent job... wink

no problem

@mokhet:
do you have a webserver running or do you access xinha through file:///?
the i18n will only work if you have a webserver! (some plugins will have the same problem)

mokhet · 2005-04-11 17:02:26

niko wrote:

do you have a webserver running or do you access xinha through file:///?
the i18n will only work if you have a webserver! (some plugins will have the same problem)

dont worry niko, i'm using a webserver and everything working fine. But yup, it's what i've noticed with the new system, not translated anymore with local files. I think this should be said somewhere in the documentation.

Another question for the incoming FAQ ? (without grammar errors of course

Q: I have done everything explained in newbie document but the texts are always in english, how can i use my language file ?
A1: Make sure your language file is up to date.
A2: The translation system (i18n) will only work if you have a webserver.

niko · 2005-04-12 14:19:16

i think you need a webserver not only for the new i18n, the plugins are loaded the same way!
(at least if you follow the newbie-guide and use if(!HTMLArea.loadPlugins(xinha_plugins, xinha_init)) return; )

so, another question could be:
Q: Do i need a webserver or can I use the script through file:/// ?
A: You need a webserver, else plugins and i18n won't work

(actually i don't know if this is right but i think so )

gogo · 2005-04-13 05:20:45

Yes, that's now correct, since we use XMLHTTPRequest for more and more things, and it's XML-HTTP-Request, not XML-FILE-Request I don't see it as a problem though, because you obviously need a webserver to do anything useful with Xinha anyway.

Xinha Discussion Forum

Announcement

#1 2005-04-04 05:20:14

new i18n-system

#2 2005-04-04 10:01:48

Re: new i18n-system

#3 2005-04-05 06:48:30

Re: new i18n-system

#4 2005-04-05 09:26:12

Re: new i18n-system

#5 2005-04-05 10:39:13

Re: new i18n-system

#6 2005-04-05 12:27:56

Re: new i18n-system

#7 2005-04-06 04:57:06

Re: new i18n-system

#8 2005-04-06 06:23:55

Re: new i18n-system

#9 2005-04-06 06:58:01

Re: new i18n-system

#10 2005-04-06 07:23:04

Re: new i18n-system

#11 2005-04-06 08:25:23

Re: new i18n-system

#12 2005-04-06 09:11:39

Re: new i18n-system

#13 2005-04-07 02:42:34

Re: new i18n-system

#14 2005-04-08 02:27:11

Re: new i18n-system

#15 2005-04-08 02:58:01

Re: new i18n-system

#16 2005-04-08 03:50:42

Re: new i18n-system

#17 2005-04-09 03:28:16

Re: new i18n-system

#18 2005-04-09 03:58:04

Re: new i18n-system

#19 2005-04-09 07:19:41

Re: new i18n-system

#20 2005-04-09 10:10:00

Re: new i18n-system

#21 2005-04-11 03:14:05

Re: new i18n-system

#22 2005-04-11 17:02:26

Re: new i18n-system

#23 2005-04-12 14:19:16

Re: new i18n-system

#24 2005-04-13 05:20:45

Re: new i18n-system

Board footer