Talk: DupImageList

Bugs

Links break when the file name contains an apostrophe (everything from the apostrophe on is removed). Also, it should do something when it's done (at least hide the spinner). ~Bobogoobo (talk) 00:59, May 7, 2013 (UTC)

TVD & TO Wiki Duplicates

Good day.

We have problems in TVD & TO Wiki, with the source to reveal the duplicate images.

A few days ago (3 or 4, I do not remember exactly) the source worked fine, but now, something happened that the source is not working, doesn't show duplicates. I researched on MediaWiki pages of TVD & TO Wiki, but I couldn't find anything that could have caused the interruption of the operation of this source.

Is it possible to have transferred the images from one server to another has to do with this?

I uploaded the same image twice, but with a different name to see if it showed as duplicates (yesterday), and these did appear, but all the others that appeared last week, no longer appear.

Thanks for your help and time.

Noctis Lux Kaelum (talk) 19:26, November 21, 2013 (UTC)

Bug Revisit

I'd like to bring back up the bug posted by Bobogoobo which has been left unaddressed to this date. The script does not know when to stop looking for duplicate images, and/or the spinner/loading icon never stops. This seems like it would be a pretty simple thing to fix for the multitude of brilliant minds we have around here. Can someone please take a look at the code and try to get that bug resolved? I've looked through it myself but I'm still very inexperienced when it comes to this level of coding, and I don't fully understand how it works. I couldn't spot any issues with the loading icon though, so if I had to guess, I'd say it might have something to do with the way it searches for duplicates... Maybe a MediaWiki upgrade broke it or something. —RyaNayR ^{(talk • contribs)} 02:08, January 3, 2014 (UTC)

I entered a fix for it, but I don't have a wiki with small number of images to test it. So that's something you would have to check and say.--~UltimateSupreme 13:40, January 4, 2014 (UTC)

I uploaded some duplicate images onto my wiki to test it out. They show up using the old code, but with the new version they don't show up. Something broke. However, I can confirm that the fix for the spinner is working great.

Here is my wiki's Duplicate Images page in case you want to see for yourself: w:c:courage:MediaWiki:DuplicateImages. That page should show four duplicates, but it doesn't. I am receiving no errors in my java console either. —RyaNayR ^{(talk • contribs)} 10:23, January 6, 2014 (UTC)

There is something wrong with your version. Just import the one here and try; its working--~UltimateSupreme 14:30, January 6, 2014 (UTC)

Oh you're right. I could have sworn I had changed 'MediaWiki:DupImageList.js' to 'u:dev:DupImageList/code.js'. I must not have saved it or something. So yeah, it's working. —RyaNayR ^{(talk • contribs)} 17:48, January 6, 2014 (UTC)

Proposition

I'm wondering if we should rewrite this article to recommend creating the list-page in the MediaWiki namespace. There's no reason for standard users to have access to edit that page on most wikis. —RyaNayR ^{(talk • contribs)} 08:19, January 14, 2014 (UTC)

Project ns is also fine. It's really upto the admins what they want.--~UltimateSupreme 08:25, January 14, 2014 (UTC)

Yeah I guess it's fine since the Project ns doesn't allow anonymous users to edit. Somehow that fact eluded me. —RyaNayR ^{(talk • contribs)} 08:47, January 14, 2014 (UTC)

Code review

This is just a note that there's a code review on DupImageList/code.js?oldid=32400 at Thread:7956. --Saftzie (talk) 22:56, October 24, 2015 (UTC)

Just another note, belated, that the review completed Oct 2015. --Saftzie (talk) 02:52, June 19, 2016 (UTC)

Script Analyzing Videos

I'm surprised no one's mentioned this yet, but the script is rather misleading for videos as it looks (seemingly) at the video thumbnail and doesn't/can't look at audio or playback length. I've dealt with a large number of videos and the majority of the time they aren't actually duplicates (examples, other wikis as well). Since this isn't a DupFileList, but an image list, perhaps videos should be ignored altogether, if possible, or be placed into a separate list (or a JS parameter to allow for the script to ignore videos). The scripts behavior with videos isn't at all helpful. DEmersonJMFM 15:27, November 1, 2015 (UTC)

I'm not sure there's a way to check if a file is a video or gif/jpg/png, other than by name. Duplicates are identified based on a "fingerprint" (really a hash of the image, I believe SHA-1). Videos just have a stub of some sort in the database. Considering the number of videos considered to be duplicates, the stubs don't vary as much as images would. So basically we need a way to tell the difference between a video and an image. I'm open to suggestions. --Saftzie (talk) 02:52, June 19, 2016 (UTC)

Files uploaded from Youtube and such don't generally have a file extension. In fact with the exception of redirects and external videos (probably a wikia hack) all other content in that namespace must have an extension. So it is easy to exclude all them. The only other alternative would be an expensive search that fetches the file metadata. But that would only provide information about the file source, not whether it is a duplicate. Dessamator (talk) 08:59, June 20, 2016 (UTC)

Done. If I missed a file extension, it'll be hard to know, since anything without a known extension gets skipped. --Saftzie (talk) 22:48, June 23, 2016 (UTC)

FWIW, I've updated to get the file extensions from the environment, so any changes to allowed types on any wiki should be covered automatically. --Saftzie (talk) 16:29, November 21, 2016 (UTC)

Raw text file.

Hi, is there anyway that this script could be used to produce a list of duplicate images that could be put into a txt document and ran by pywikipedia to delete these images? Similar to how a script, cannot remember which, can display a full list on Special:UnusedFiles of unused files. --Original Authority ^(talk) 22:12, January 2, 2016 (UTC)

It could probably be done. The easiest way would be to specify the output format in the div, something like <div id="mw-dupimages" data-options="flat">. Instead of a bulleted list, it could populate a textarea. You'd still want to know which files were the "master" copies, I presume. --Saftzie (talk) 02:52, June 19, 2016 (UTC)

Outside Wikia

This script is not working outside Wikia wikis, for example on WMF wikis (like Wikipedia). You can test it on http://test.wikipedia.org/ or http://test2.wikipedia.org/ --Steaua10 (talk) 11:39, November 21, 2016 (UTC)

It's not intended to work anywhere else. Most, if not all, scripts on dev are for Wikia only. Some might work elsewhere, but why would you think they should work elsewhere? --Saftzie (talk) 14:59, November 21, 2016 (UTC)

...Because they are using the same engine - MediaWiki. Only versions are different. --Steaua10 (talk) 15:46, November 21, 2016 (UTC)

Wikia forked their engine at MW 1.19, so...

1. Different versions means different versions. There are differences.
2. It's not even MW 1.19 any more. Wikia has stated they intend not to re-norm to another version of MW ever.

--Saftzie (talk) 16:00, November 21, 2016 (UTC)

Format

Why the "main images" are headings? Wouldn't it be better to have the list in this format instead?

* Image 1
** Duplicate 1
** Duplicate 2
* Image 2
** Duplicate 1
** Duplicate 2

The headings make the file names weird. leviathan_89 15:12, November 10, 2017 (UTC)

Mostly because the original authors wrote it that way. Alternate formats could impact existing workflows. There's no way to know who would be impacted by a change or how. Saftzie (talk) 00:31, November 11, 2017 (UTC)

Cross File Searching?

I recently added this code to the Alternate History Wiki. Just wanted to say it's absolutely great that this program exists and we thank you.

However, I've noticed that the duplication list only shows similar files of the exact same file type. By this, I mean it only shows duplicate JPG files with other JPG files, and PNG files with other PNG files (etc.).

Is there a way to allow the code to include cross file searching (i.e., compare JPGs with PNGs)? If not, may I suggest this feature be considered.

Thank you for your time. -- NuclearVacuum 03:09, July 18, 2018 (UTC)

I've also just noticed that the list doesn't show duplicates unless they are 100% the same. It appears to ignore duplicates that are of different file types and of different dimensions (size). May I request that this be looked into. Thank you. -- NuclearVacuum 17:53, July 18, 2018 (UTC)

Query Continue and UCP

Just a FYI. This script may break when the upcoming Unified Community Platform (UCP) arrives as query continue semantics have changed in the latest MediaWiki version. Easiest fix is to add the rawcontinue parameter to the appropriate queries to force the use of the current query continue response. Aspallar (talk) 17:53, December 27, 2019 (UTC)

UCP incompatibility

This extension does not work with UCP. It throws the following error:

load.php?debug=false&lang=en&modules=ext.fandom.ImportJs%2Csite&skin=oasis&version=1v4d7yw:formatted:1112 Uncaught (in promise) DOMException: Failed to execute 'setAttributeNS' on 'Element': 'http://www.w3.org/2000/svg' is an invalid namespace for attributes.

which occurs at

svga.forEach(function(value, name) {
    spinner.setAttributeNS(svgns, name, value);
});

- FDekker ^(talk) 14:09, November 10, 2020 (UTC)

A complete rewrite of the script is being planned at this moment. That was not supposed to happen, and I will have to make a fix to it. Ultimate Dark Carnage (talk) 15:55, November 10, 2020 (UTC)

Understandable. I think it would be worthwhile to mark the script as "Awaiting UCP compatibility" on Dev Wiki:UCP to inform users of this issue. - FDekker ^(talk) 20:19, November 10, 2020 (UTC)