Comments on: My solution to the counting sets challenge http://www.algorithm.co.il/blogs/challenges/my-solution-to-the-counting-sets-challenge/ Algorithms, for the heck of it Tue, 21 Jun 2011 21:07:08 +0000 hourly 1 http://wordpress.org/?v=3.1.3 By: lorg http://www.algorithm.co.il/blogs/challenges/my-solution-to-the-counting-sets-challenge/#comment-226 lorg Mon, 07 Sep 2009 22:50:39 +0000 http://www.algorithm.co.il/blogs/?p=325#comment-226 rouli: Regarding every two sets' union being equal to the union of all sets - well, fortunately that doesn't happen in my input. I forgot to mention that before I implemented the algorithm I did various "dry tests", of estimating the the number of containment tests. In the general/worst case (without 'max_appear'), where the input set is also contained in many set, it really isn't good. It's very easy to kill this algorithm, but for 'sane' input it performs well enough. As I said - there are many other drawbacks. For example, the set group may be ordered in such a way that the tree built from it will be very inefficient later on. My (theoretical) cheap shot at this was to randomize the order. I thought of some more drawbacks, but enough for now. Especially since it works :) rouli:
Regarding every two sets’ union being equal to the union of all sets – well, fortunately that doesn’t happen in my input.
I forgot to mention that before I implemented the algorithm I did various “dry tests”, of estimating the the number of containment tests. In the general/worst case (without ‘max_appear’), where the input set is also contained in many set, it really isn’t good. It’s very easy to kill this algorithm, but for ‘sane’ input it performs well enough.

As I said – there are many other drawbacks. For example, the set group may be ordered in such a way that the tree built from it will be very inefficient later on. My (theoretical) cheap shot at this was to randomize the order. I thought of some more drawbacks, but enough for now. Especially since it works :)

]]> By: lorg http://www.algorithm.co.il/blogs/challenges/my-solution-to-the-counting-sets-challenge/#comment-225 lorg Mon, 07 Sep 2009 22:42:39 +0000 http://www.algorithm.co.il/blogs/?p=325#comment-225 Rani: It's good to see the proper name for the "soldier-testing" algorithm, thanks! I didn't understand though if you can adapt/find a better solution for the challenge. I'll be happy to hear what you think - especially if it reduces the running time of my (very real) program :) Rani:
It’s good to see the proper name for the “soldier-testing” algorithm, thanks!
I didn’t understand though if you can adapt/find a better solution for the challenge. I’ll be happy to hear what you think – especially if it reduces the running time of my (very real) program :)

]]> By: lorg http://www.algorithm.co.il/blogs/challenges/my-solution-to-the-counting-sets-challenge/#comment-224 lorg Mon, 07 Sep 2009 22:38:51 +0000 http://www.algorithm.co.il/blogs/?p=325#comment-224 rouli: That was one of the reasons I didn't implement this solution right away. If the given set is contained in all the sets, you'll do 2*n-1 checks to discover this. Before I thought of 'max_appear', I thought of other checks you can do before you run it, and then maybe run a different algorithm, maybe even the naive one. I also tried running eric and Shenberg's solution only for input sets of cardinality 1 and 2 (and for the rest run my solution), but that one intersection was enough to kill it. There's still a possibility that there are a few implementation kinks I put there that can fix it, but I doubt it. rouli:
That was one of the reasons I didn’t implement this solution right away. If the given set is contained in all the sets, you’ll do 2*n-1 checks to discover this. Before I thought of ‘max_appear’, I thought of other checks you can do before you run it, and then maybe run a different algorithm, maybe even the naive one.

I also tried running eric and Shenberg’s solution only for input sets of cardinality 1 and 2 (and for the rest run my solution), but that one intersection was enough to kill it. There’s still a possibility that there are a few implementation kinks I put there that can fix it, but I doubt it.

]]> By: rouli http://www.algorithm.co.il/blogs/challenges/my-solution-to-the-counting-sets-challenge/#comment-223 rouli Mon, 07 Sep 2009 20:17:11 +0000 http://www.algorithm.co.il/blogs/?p=325#comment-223 Actually, I pondered about this method, and dismissed it once I've thought about the scenario where the union of every two sets in the original list of sets equals the union of all the sets in the original list. In that case, you'll have to go all the way down, doing more comparisons than in the naive algorithm. (Obviously it's solved by introducing max_appear) Actually, I pondered about this method, and dismissed it once I’ve thought about the scenario where the union of every two sets in the original list of sets equals the union of all the sets in the original list.
In that case, you’ll have to go all the way down, doing more comparisons than in the naive algorithm.
(Obviously it’s solved by introducing max_appear)

]]> By: Rani http://www.algorithm.co.il/blogs/challenges/my-solution-to-the-counting-sets-challenge/#comment-222 Rani Mon, 07 Sep 2009 19:28:09 +0000 http://www.algorithm.co.il/blogs/?p=325#comment-222 The problem you're referring to is called Combinatorial Group Testing, and has efficient* solutions even when: (i) more than one soldier is sick (say, at most k out of n), and (ii) you want to decide in advance which samples get sent to the lab, instead of waiting log(n) roundtrips (this property is called non-adaptiveness). For the case of k=1 you can pick the unions to check as follows: union number i has all sets whose index j satisfies (j>>i)&1==1. * Thats O( k^2 log(n/k) ). The problem you’re referring to is called Combinatorial Group Testing, and has efficient* solutions even when:
(i) more than one soldier is sick (say, at most k out of n), and
(ii) you want to decide in advance which samples get sent to the lab, instead of waiting log(n) roundtrips (this property is called non-adaptiveness).

For the case of k=1 you can pick the unions to check as follows:
union number i has all sets whose index j satisfies (j>>i)&1==1.

* Thats O( k^2 log(n/k) ).

]]>