Talk: I18n-js

Walkthrough

Could you please include a walkthrough example like this?

mw.hook('dev.i18n').add(function (i18n) {
    i18n.loadMessages('I18n').done(function (i18n) {
        console.log(i18n.msg('refreshHover').parse());
    });
});

Agent Zuri Profile Message Wall Blog 10:22, July 3, 2017 (UTC)

I'm sceptical of providing a simple, easy to use snippet as to use this you need to be proficient at using JS which also requires reading documentation at various times.—Preceding unsigned comment added by Cqm (talk • contribs) . Please sign your posts with ~~~~!

To me your documentation is not that clear so I had to find out on my own how it worked and that is what I think the initial point where I can start to work with your script. Agent Zuri Profile Message Wall Blog 16:05, July 5, 2017 (UTC)

list of all available messages

Could you provide a list of all available messages as well (maybe using DPL or something similar)?
Thanks in advance, Agent Zuri Profile Message Wall Blog 10:25, July 3, 2017 (UTC)

DPL mightwould get you the pages with the messages, but not the JSON stored within them. In truth, I'm not entirely sure what you're asking for or what the use-case for this is. cqm 06:51, 5 Jul 2017 (UTC)

It's not about getting all the code. It's just do give a user an overview of all the translations which are available. Of curse you could simply use a category for this case as well. Agent Zuri Profile Message Wall Blog 16:05, July 5, 2017 (UTC)

List of added translation languages

Hi,
could you display the list of languages which already have translations in I18nEdit in the infobox on the page of the specific script as well?
Agent Zuri Profile Message Wall Blog 09:58, August 14, 2017 (UTC)

Highlight missing translation

In the language translation table, messages without any translation provided should be highlighted in some way. leviathan_89 22:40, December 10, 2017 (UTC)

Translations bugs

I see a few weird things with the localization of this script:

• Some of the message descriptions are not in English.
• For Italian, I see two extra messages that are not present in English (and have no values), metadata-editore-text and title-editore.
• How do I localize the warning message displayed on JSON pages? leviathan_89 12:20, December 18, 2017 (UTC)

• They have been removed. I might add the descriptions later.
• I removed the invalid values. They seem to have been added in this gargantuan edit and I have no idea why. I'll take a look at it if it happens again and in an edit that was actually made by the editor.
• You can't, as far as I know. It loads content from MediaWiki:Custom-I18nEdit-notice. In order to provide support for translating the messages, you'd have to edit the script code. -- Dorumin 01:31, January 3, 2018 (UTC)

Actually, you can already localize the notice. Just create a subpage of MediaWiki:Custom-I18nEdit-notice like I did here. -- Cube-shaped garbage can 08:26, January 3, 2018 (UTC)

Thank you both! leviathan_89 10:33, January 3, 2018 (UTC)

Editor improvements

Some improvements that could be made based on me trying to use the editor:

• Add a button to create a new i18n.json page from Special:Blankpage/i18nedit, preferably with a check to ensure the script exists, e.g. does NAME, MediaWiki:NAME.js or MediaWiki:NAME/code.js exist?
• When editing message descriptions, the descriptions column is confusingly named "Translation". Why not call it "Message Description" instead so you can tell what it's meant to be?
• The edit summary box isn't clearly identified as being such. It'd be nice to have a label on the box to show what it's used for.

cqm 22:29, 29 May 2018 (UTC)

\/\/

https://dev.wikia.com/index.php?title=MediaWiki%3ACustom-DiscordModule%2Fi18n.json&diff=79427&oldid=79426
~[Quəntumiið the Wise] 19:35, June 3, 2018 (UTC)

Special:Diff/92314 removed support for // inline comments, which means slashes should no longer need escaping! - OneTwoThreeFall ^talk 07:04, October 9, 2018 (UTC)

Plural

The script should definitely support plural natively in any form. Why? Because most devs will not (or can't) implement wikitext support in their codes, what generate problems with some languages, e.g. with Polish. — Rail _{(my talk | my contributions)} 13:46, July 5, 2018 (UTC)

Plural support was added with Special:Diff/90727! - OneTwoThreeFall ^talk 07:01, October 9, 2018 (UTC)

mediawiki.api

It turns out the mw.Api constructor accepts JSONP datatype (example in hljs.registry.init. The main ResouceLoader module appears to be available in any MediaWiki version, so abandoning $.ajax won't be detrimental to script portability cross-platform. • speedy • 🔔︎ • 🚀︎ • 17:10, October 8, 2018 (UTC)

Yep, mw.Api will accept anything $.ajax will, since it's just a $.ajax wrapper with a few handy MW-specific features. Considering this isn't using any of those features, though, I can't really see a benefit of using it over $.ajax, plus it'd require loading another dependency (the mw.Api module isn't loaded by default). - OneTwoThreeFall ^talk 06:38, October 9, 2018 (UTC)

Messages as public class constructor

A lot of people on Dev Wiki use a preload method and counter to load dependencies. But this doesn't normally fit well with loading i18n messages.

I'd like if i18n function was a public constructor (perhaps window.dev.i18n.Messages), so that I could write code like this:

this._preload = function (dep) {

    if (dep === window.dev.i18n) {
        window.dev.i18n
            .loadMessages('NAME')
            .then(this._preload.bind(this));
        return;

    } else if (
        (window.dev || {}).i18n &
        dep instanceof window.dev.i18n.Messages
    ) {
        this.i18n = dep;
    }

    if (++this.__loaded === Object.keys(this._libs).length) {
        this._init();
    }

}

It looks more complicated, but the speed increase is quite significant for say, EmoticonsWindow.

• speedy • 🔔︎ • 🚀︎ • 14:48, December 14, 2018 (UTC)

Don't know how others feel, but I don't think making major structural changes to i18n-js is a good idea - it needs to stay solid and stable now it's so widely used. Looking at current EmoticonsWindow code, perhaps something like below (using the existing bind calls to pass through the library name) could serve your use case?

this._preload = function (libName, dep) {
    if (libName === 'i18n') {
        window.dev.i18n
            .loadMessages('NAME')
            .then(this._preload.bind(this, 'i18n-messages'));
        return;
    }

    if (libName === 'i18n-messages') {
        this.i18n = dep;
    }

    if (++this.__loaded === Object.keys(this._libs).length) {
        this._init();
    }
}

this._hook = function (libName) {
    mw.hook('dev.' + libName).add(this._preload.bind(this, libName));
};

Object.keys(this._libs).forEach(this._hook.bind(this));

- OneTwoThreeFall ^talk 06:30, December 15, 2018 (UTC)

Agreed about structural changes. It's a bit of shame i18n messages weren't designed as a true ES5 instance of anything (also for readability!). But if it works well by scope atm, I'm not sure I can sell the idea any better :p

That's also quite a neat alternative for i18n message caching. Didn't consider Function.prototype.bind's support for multiple arguments.

• speedy • 🔔︎ • 🚀︎ • 17:11, December 15, 2018 (UTC)

I was just looking back at i18n-js code history, and noticed it started out as prototype-based but was then changed to the current structure. I don't know the reasons behind the change but thought you may be interested! - OneTwoThreeFall ^talk 10:42, December 20, 2018 (UTC)

Definitely intriguing. My leading theory: the change keeps certain language and message parameters private when returned. They can only be modified through the factory export's methods, preventing their mutation from a Messages object.

Still agreed this approach is safer than the prospect of changing it back :)

(Also interesting - using Message.prototype method & .bind inside the factory method, these private variables could theoretically be supported.. though it would be a little questionable for readabiliy and not really worth it).

• speedy • 🔔︎ • 🚀︎ • 11:08, December 20, 2018 (UTC)

PS: The Wikipedia page seems to back this up ("relies on inheritance.." etc). And it lists some other benefits :)

• speedy • 🔔︎ • 🚀︎ • 12:29, December 20, 2018 (UTC)

Agreed, seems a likely theory. Much easier to stick with what we have now too! :) - OneTwoThreeFall ^talk 13:38, December 20, 2018 (UTC)

Overriding messages may not work

The current implementation uses a load guard that checks whether window.dev.i18n is defined. This is not quite compatible with the documented instructions for overriding messages. For site-wide changes (via Wikia.js or Common.js), defining window.dev.i18n.overrides would prevent I18n-js from subsequently bootstrapping (assuming the common case where an ImportJS script first pulls in I18n-js). puxlit (talk) 15:01, January 23, 2019 (UTC)

Ouch, guess this feature wasn't used much else it surely would've been noticed by now! I've changed it to check window.dev.i18n.loadMessages instead. - OneTwoThreeFall ^talk 05:13, January 24, 2019 (UTC)

Thanks for the quick fix! puxlit (talk) 04:33, January 25, 2019 (UTC)

update information

Hey folks, could you display, when there are languages which lag behind? Let's consider the following scenario:

1. The script author adds two entries to qqx
2. Users add five translations with two entries each
3. The script author updates qqx and his own language with three more entries
4. All the other languages now have two entries and three are missing.

What do you think about that?

Best regards,
Agent Zuri Profile Message Wall Blog 18:14, February 12, 2019 (UTC)

Problems with Edge

I found recently that the editor does not filter languages when going to add new language translation and the search box for filtering results has some text. Only happens in the Edge browser, not others. If it would be fixed, it may be nice since I can’t stand scrolling to search a language. HM100 17:59, March 8, 2019 (UTC)

Can't reproduce on Windows 10 + Edge 18 (via BrowserStack). Could you clarify which versions you're on? EDIT: Per IM follow-up, it appears this is related to I18n-js not gracefully handling localStorage QuotaExceededErrors. puxlit (talk) 13:02, March 9, 2019 (UTC); edited 13:20, March 9, 2019 (UTC)

Depends on your definition of "graceful", I suppose :). I18n-js wraps all calls to localStorage in a try-catch block and, if data can't be loaded, falls back to loading i18n directly so any error thrown shouldn't cause I18n-js to fail. Can't test Edge right now, but I'd assume the issue here is somewhere in I18nEdit's code. - OneTwoThreeFall ^talk 07:07, March 10, 2019 (UTC)

Okay, I gave Edge a try and filled localStorage with junk to get the error - the issue is due to this I18nEdit function:

filterLanguages: function(e) {
    var val = e.target.value.toLowerCase(),
    filters = val.split('|');
    localStorage.setItem('i18nEdit-filter', val);
    $('.I18nEditNewLanguageList .noexist').hide().filter(function() {
        var $this = $(this),
        text = $this.text().toLowerCase(),
        indexes = 0;
        filters.forEach(function(lang) {
            indexes += text.indexOf(lang);
        });
        return indexes !== -filters.length;
    }).show();
},

The localStorage.setItem line throws, and therefore the code after never runs. I've moved that line to the end so at least the filtering will work before the error is thrown, but ideally I18nEdit would check localStorage is actually usable before trying to use it (there's a bunch of edge cases that mean using it even in modern browsers can have issues). - OneTwoThreeFall ^talk 11:01, March 10, 2019 (UTC)

Caching improvements

Current caching method used for translations isn't very good. Actually, it's bad. We don't need to store all available languages for each script in cache, so after a Discord talk with Kocka and others I'd want to suggest improved logic for cache:

• Save language that user set in preferences (mw.user.options.get('language') variable) in cache.
• Check for languages' fallbacks (usually it's English, but sometimes other languages as well) and cache them, too.
• Treat ?uselang as a debug mode so ignore caching for mw.config.get('wgUserLanguage') since usually its used for testing or other temporary usage.
• Optionally implement some settings variable to let users choose which languages do they want in cache.

Method used now uses much more space in users' browsers than really necessary, what is really bad because our codes are used by thousands, maybe even millions of users across Fandom everyday. We should respect their disk space.

RAIL _{my talk} 14:37, October 16, 2019 (UTC)

There are currently three language modes in I18n-js: user language mode, content language mode and custom language mode. What Rail mentioned above would work for returning translations in the user language mode. Additionally caching strings that would appear in the content language mode should not be an issue, however, there is still the custom language mode (i18nd.useLang(...) / i18nd.inLang(...)) that we would need to account for when caching languages.

Adding a switch to enable the custom language mode for specified languages in the parameter to loadMessages() sounds like it would solve the problem, but only if the scripts that previously used the custom language mode add that option into their configuration, so this would potentially be a breaking change.

On the positive side, not many scripts use the custom language mode that I know of. I only remember LastEdited using it myself, we are able to search the code dump for other scripts on Dev that use it too, and we can probably assume no scripts outside of Dev that are importing I18n-js are using that option, too.

-- Cube-shaped garbage can 14:56, October 16, 2019 (UTC)

If wasting localStorage space is the primary concern, I'd tentatively suggest switching over to the Cache API (noting that IE11 and iOS Safari are the two main stragglers).

If localStorage is a must, then one approach to facilitating useLang(…)/inLang(…)/etc. without API changes is to follow Rail's proposed strategy, but keep the full set in memory and note down (via another key) that we're storing a subset of the full messages. If a caller then tries to fetch a message for a language that we wouldn't have cached as part of the subset, we'll "upgrade" and store the full set of messages. puxlit (talk) 00:57, October 17, 2019 (UTC); edited 01:12, October 17, 2019 (UTC)

I would disagree that the current caching method is bad. It caches what gets loaded, the JSON is (as far as I can tell) minified and we only cache for 2 days. If cache space were not an issue, I'd leave it as is.

Having said that, I'm open to improvements. At the moment we have an in memory cache, but maybe we could add something to only cache what gets used in localStorage? That results in more small writes on first use, but hopefully isn't more taxing once we've got a cache.

The above boils down to checking if we can load messages from localStorage and putting them in the memory cache. If we get a request for a message we don't have, we load via AJAX and mark that we've done so (and reset the cache time to 0). Every time we use a message not in localStorage, we write it there with an appropriate cache duration and maybe extend the current 2 days duration.

Thoughts? cqm 19:51, 21 Oct 2019 (UTC) 19:51, October 21, 2019 (UTC)

Been meaning to look at this for a while - I've created a preview version of i18n-js at MediaWiki:I18n-js/optimise-cache.js that basically implements what's been discussed here.

Basically, it will preprocess the languages in wgUserLanguage and wgContentLanguage by calculating all the fallback messages (e.g so if user lang is pt-br, it will collect the pt-br messages, with any missing ones filled by pt, and anything still missing by en), dump everything else (including fallback languages, since they're already computed), and save just user and content language messages to the localStorage cache.

This does mean it drops support for useLang() and inLang(). I personally don't see the use for setting a different language for just one specific script or message, but I'd be interested in use cases for that feature. If there is one we can always add a useLang option to loadMessages() as KockaAdmiralac suggested.

Interested in feedback and thoughts! - OneTwoThreeFall ^talk 11:46, July 9, 2020 (UTC)

I've done a search of the wiki: no code uses inLang(). Three scripts do use useLang(): MediaWiki:AdvancedOasisUI/code.js:160, MediaWiki:LastEdited/code.js:143, and MediaWiki:RailWAM/code.js:137. All three use it within their i18n-js init call to set a language given in script options.

Considering this, I'd like to drop support for useLang() and inLang() and replace it with a loadMessages() option (like i18n.loadMessages('Script', {language: 'xx'})). This will be both compatible with an optimised cache and support all current useLang() usage on Dev Wiki.

- OneTwoThreeFall ^talk 05:20, July 11, 2020 (UTC)

Little update: I18n-js now supports a language option, and AdvancedOasisUI, LastEdited, and RailWAM have been updated to use it. There are now no scripts on the wiki using useLang() or inLang(). - OneTwoThreeFall ^talk 05:49, July 19, 2020 (UTC)

It's been six weeks and no one had brought up any issues or comments, so I've gone ahead and updated MediaWiki:I18n-js/code.js with support for an optimised cache, and removed support for useLang() and inLang(). - OneTwoThreeFall ^talk 13:42, August 25, 2020 (UTC)

Quentum recently pointed out to me how scripts doing cross-wiki edits (in his case, HotCat) may need to use summaries based on languages of wikis being edited, which are not necessarily in either the user language or content language. Could we bring back the functionality inLang() had for this use case, by requiring the developer to explicitly mark messages for which all translations need to be cached in loadMessages() options? -- Cube-shaped garbage can 02:24, November 21, 2020 (UTC)

Interesting use case! Seems the current version doesn't use I18n-js, but if you're testing, here's a version that adds a cacheAll option.

Right now, this option's value can be either an array of message names that should be kept for all languages, or true, which just disables the optimised cache (a much simpler code change). Right now I'm inclined to just add the cacheAll = true option as this sort of requirement seems rare, but let me know what you think! - OneTwoThreeFall ^talk 04:28, November 21, 2020 (UTC)

That looks good, thanks! I'm not the developer of the I18n-js-localized version of HotCat but I'll pass it on. cacheAll = true sounds like it could potentially be needed for some use case, but I wouldn't mind it not being implemented until we get a solid use case for it. -- Cube-shaped garbage can 02:26, November 22, 2020 (UTC)

The developer reports that the option works for them on w:c:starwars:User:01miki10/common.js, but since many other scripts are loading I18n-js the cacheAll version is prevented from loading (instead, the stable version of I18n-js loads). Do you think this could be brought to stable now? -- Cube-shaped garbage can 20:55, November 23, 2020 (UTC)

Can't see why not!. - OneTwoThreeFall ^talk 09:21, November 28, 2020 (UTC)

I know this thread has gone for long enough but another issue arised with the current implementation of this - uselang does not act as a debug mode as Rail originally suggested, so I18n-js does not display translations in the language in uselang but rather in English, even though translations in that language exist. What also comes to mind are users permanently changing their preferred language in preferences - should I18n-js cache the user's preferred language (like it caches the last cache timestamp) and re-cache when it detects a user language mismatch (when uselang isn't used)? I can implement both of these changes if they sound fine. -- Cube-shaped garbage can 14:41, December 5, 2020 (UTC)

Good spot! Support for user or content language changing already exists (it's a situation I made sure to test, both with uselang in use, and with using a cache created on, for example, example.fandom.com/fr/ with example.fandom.com/es/), but somewhere along the line I broke it and obviously didn't notice, sorry about this. All the more reason to get proper tests in place I think!

Should be fixed with this change, which makes sure I18n-js won't optimise messages loaded from the cache - a process which creates a property for each wanted language - which then fooled cacheIsSuitable() (which would check the user and content languages exist, and cause a load from the server if not) into thinking all was good. Let me know if it's still working not as you expect with this change. - OneTwoThreeFall ^talk 09:13, December 6, 2020 (UTC)

zh support?

So I just noticed that the zh and zh-hant translations in my json file don't show up in the editor. I think I'm supposed to rename zh-hant to zh-tw (right?), but what confuses me is "zh" being blacklisted in editor.js. The editor.js code refrences here for what is blacklisted, but that link explicitly states that "zh" is supported. I'm not sure if maybe it used to be supported and now isn't, or maybe it's it should be removed from the blacklist now. Don't know enough about it to make a choice.

Also, in code.js a bunch of things have fallback support to zh-hans, yet not only is zh-hans blacklisted in editor.js, but in code.js I don't see any fallback support for zh-hans to something else; I might be missing something, but figured I'd mention it while I'm here. ☙Fewfre​☎ 🔎 K​🧀​18:53 Mon, 02 Nov 2020

The second part is done and pending approval. I'm not sure why is zh blacklisted, but I think it's probably because other language codes need to be used instead and they'll fall back to zh. -- Cube-shaped garbage can 23:52, November 2, 2020 (UTC)

Alright, thanks; And makes sense, I'll try to ask the original translator what "zh" should be changed to then. ☙Fewfre​☎ 🔎 K​🧀​02:16 Tue, 03 Nov 2020

Array/Object support in Translation data?

Question: In translation messages (Translation JSON data); does this script support arrays/objects as values for the data? I know the editor does not support the mentioned data values.

~☙Thundercraft5|💬|🔎||GuestbookLeft on 2:21 AM Nov 04 (Wed) 2020

Even if it could possibly load them, I don't see what they could be used for. What is your intended use case? -- Cube-shaped garbage can 12:36, November 4, 2020 (UTC)

My intended case is a object for default rename reasons. This message, if it were to exist, would be loaded when the user would not supply the same object via a script configuration. My suggestion for the editor behavior as is follows: each language would contain this message object where some key values are strings would be translatable, and any keys in the object's fields would follow the same protocol recursively (I could add this data manually but the maintenance would be horrible).

~☙Thundercraft5|💬|🔎||GuestbookLeft on 7:31 AM Nov 05 (Thu) 2020

You could achieve that with:

var defaultReasonKeys = ['standardization', 'vandalism', 'other'], defaultReasons = {};
defaultReasonKeys.forEach(function(key) {
    defaultReasons[i18n.msg('reason-label-' + key).plain()] = i18n.msg('reason-value-' + key).plain();
});

and then separate your translations to reason-label-standardization, reason-label-vandalism, reason-label-other, reason-value-standardization, reason-value-vandalism, reason-value-other.

-- Cube-shaped garbage can 18:50, November 8, 2020 (UTC)