Gossip Glomers: Broadcast Questions/Clarification

1

Preface: I have never worked with any kind of distributed system before, nor with Go, so some number of these misunderstandings are quite likely my own alone.

I have worked through Challenge #3, but after reaching the end I am fully convinced that I have “missed” large portions of the lesson. For my own sake and others, I’d like to just ask for clarification on some of the various specifications/evaluations. I don’t want to continue on to Challenge #4 until I’m sure I’ve absorbed the intended lessons from Challenge #3

3b

Your node should propagate values it sees from broadcast messages to the other nodes in the cluster.

I just handled each “broadcast” by doing a Send to every other node using a new message type (I called it “sync” – not “broadcast”). Then in that handler, I never need to send that message onward but instead just record it, so there was very little to do.

I’m not convinced that I did the expected thing here, because I can see some users in a different thread on this site have done it using only the “broadcast” type. In the event that I did not do what you expected, is there a way to rewrite the prompts to better explain what you want?

3c

Now, since I had no idea what I was doing, at the time, my solution for 3b seemed reasonable to me, but here is the first point where I started to get suspicious that I was not on the intended path.

This only required two small changes to my code from #3b. First, instead of sending just the new broadcast number to the other nodes, I sent the entire list of numbers that node knew about to every other node. And then in the handler for the “sync” message, I just needed to merge the recipient’s list of numbers with the sent list, without duplicates. Even if I was using the “broadcast” message only, like other users on the forum, I could have gone with this same approach.

Since this only took a couple minutes, I was starting to get very concerned that I had fallen off the tracks, even though my code had no problems passing the included Maelstrom test a few times in a row.

I then reviewed the specification closely in case I had missed anything. I saw this part:

Nodes should only return copies of their own local values.

My first assumption was that it just means I’m not supposed to wait to get a “read” request and then go out and ask all the other nodes before responding. But maybe you mean something different (like “copies” as opposed to “pointers”) with that and can clarify it?

3d

And by this point I was basically completely sure I wasn’t learning what you were teaching.

First, the difference in formatting between the specification and the explanation is confusing. If the goal in messages-per-operation is “30”, please show an example under the “Messages-per-operation” heading that would pass, and another that would not pass the criteria. It is still not clear to me whether the example shown is passing or not (I think not, but not sure) because you talk about 63, 126, and also 5 in the explanation paragraph.

Second, I didn’t have to make any changes to my code and from everything I can understand of this page, it was already doing better than the goal figures. I am pretty sure it is because just sending all the data with every Send is cheating, but that is just a guess. Again, I have not seen anything in the instructions that tell you not to do that, but this might be a “duh” thing for anyone who already knew one or two bits of information about distributed systems and sending messages.

3e

Basically the same complaint from 3d, I didn’t have to change anything in my code so I know I am way, way off base by now.

1 Like
2

The challenges are flexible in that you can implement them in a variety of ways. You don’t need to reuse the broadcast message type. Your approach works just fine.

The goal of challenge 3b is to simply make your single-node implementation distributed. The most interesting challenge is later on when you need to make it more efficient.

That’s the easiest approach to the problem. It’s not practical in a real-world system as the data set can get quite large. Maelstrom has some limitations in exactly what it can evaluate so it doesn’t take into account the payload size or space efficiency.

The challenges aren’t mean to be a perfect evaluation of distributed systems skill. There’s ways to hack around a toy system like Maelstrom that won’t work in larger systems. If you’re interested in challenging yourself further, I’d suggest sending individual values instead of the full data set.

Yes, it means that you should read from whatever data is currently held in-memory on the node and it should not perform an RPC to all other nodes.

I updated challenge 3b to recommend against sending all data in each message so it will hopefully make the challenges more challenging.

3

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.