remove duplicates from a group

remove duplicates from a group

[Julien Cubizolles]

I've got several nnmaildir groups with many duplicate messages sharing the
same Message-ID. I know Gnus can act on duplicate messages it receives
according to nnmail-treat-duplicates, but how to deal with duplicate
messages already in a nnmaildir group ?

Ideally, I would like to mark all the duplicate messages in a group. 

-- 
Julien Cubizolles

Re: remove duplicates from a group

[Eric Abrahamsen]

Julien Cubizolles <j.cubizolles@free.fr> writes:

> I've got several nnmaildir groups with many duplicate messages sharing the
> same Message-ID. I know Gnus can act on duplicate messages it receives
> according to nnmail-treat-duplicates, but how to deal with duplicate
> messages already in a nnmaildir group ?
>
> Ideally, I would like to mark all the duplicate messages in a group. 

No, there's nothing like that available right now. Probably we'd want it
to be a search function, rather than marking or limiting, since the
latter would require you to first open a summary buffer containing all
the articles in the group. Search would ensure that you're checking the
whole group for duplicates, and then maybe there could be a dedicated
marking function after that, for marking all but one of duplicate
messages.

But none of that exists right now!

Re: remove duplicates from a group

[Julien Cubizolles]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Julien Cubizolles <j.cubizolles@free.fr> writes:
>
>> I've got several nnmaildir groups with many duplicate messages sharing the
>> same Message-ID. I know Gnus can act on duplicate messages it receives
>> according to nnmail-treat-duplicates, but how to deal with duplicate
>> messages already in a nnmaildir group ?
>>
>> Ideally, I would like to mark all the duplicate messages in a group. 
>
> No, there's nothing like that available right now. Probably we'd want it
> to be a search function, rather than marking or limiting, since the
> latter would require you to first open a summary buffer containing all
> the articles in the group. Search would ensure that you're checking the
> whole group for duplicates, and then maybe there could be a dedicated
> marking function after that, for marking all but one of duplicate
> messages.
>
> But none of that exists right now!

Thanks, now I can stop searching for this function and maybe start
coding…

-- 
Julien Cubizolles

Re: remove duplicates from a group

[Eric Abrahamsen]

Julien Cubizolles <j.cubizolles@free.fr> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Julien Cubizolles <j.cubizolles@free.fr> writes:
>>
>>> I've got several nnmaildir groups with many duplicate messages sharing the
>>> same Message-ID. I know Gnus can act on duplicate messages it receives
>>> according to nnmail-treat-duplicates, but how to deal with duplicate
>>> messages already in a nnmaildir group ?
>>>
>>> Ideally, I would like to mark all the duplicate messages in a group. 
>>
>> No, there's nothing like that available right now. Probably we'd want it
>> to be a search function, rather than marking or limiting, since the
>> latter would require you to first open a summary buffer containing all
>> the articles in the group. Search would ensure that you're checking the
>> whole group for duplicates, and then maybe there could be a dedicated
>> marking function after that, for marking all but one of duplicate
>> messages.
>>
>> But none of that exists right now!
>
> Thanks, now I can stop searching for this function and maybe start
> coding…

It's probably easiest to start with the limiting function: once a
newsgroup is open you have access to `gnus-newsgroup-headers`, and it
should be fairly straightforward to sort them by message-id, then pick
out the article numbers of all consecutive duplicates, and pass that to
marking. Then you're halfway there!