[OAI-implementers] Handling set membership changes

[Chris Wilper]

Hi all,

So, I was looking at what it would take to get Fedora's OAI provider interface to support sets for selective harvesting and I realized that communicating set membership changes to harvesters could be trouble.

Is there a sane way to deal with this?  The only thing I can think of is to just  treat the set membership change as a record modification, but this leaves certain harvesters in the dark.  They will believe the record still exists in its prior set forever and hasn't been modified forever, unless:

  They are smart enough to recognize a setSpec change in a header, and
     a) They are lucky enough be selectively harvesting 
        on one of the new sets the record is in, or
     b) They only do incremental (not set-selective) harvesting.

-or-  

  They periodically dump everything and do a full harvest.

If I know up front that set membership is apt to change, should I:

  1) Not support sets in my provider.

  2) Support sets, but have a policy that if someone is harvesting
     from me, they agree to either:
       i)  periodically dump everything and do a full harvest, or
       ii) never do set-selective harvesting

  3) Something else

Any recommendations or opinions on this?

Thanks,
Chris