Just over a week ago I gave a talk at ETHAtlanta with regards to how endorsers could delegate endorsement to verifiers. Plenty of feedback was received and it will be incorporated into the code that will be open sourced. If you are interested you can download the deck and notes below.
The upcoming release of Go 1.7 shall move the golang.org/x/net/context package into the standard library. The move standardizes a simple interface and several simple functions that allow for straightforward concurrent processing as detailed in a 2014 Go blog post. More importantly a Context can be attached *http.Request allowing for cleaner and more concise code.
An example is a recurring pattern used in middleware or http handler processing that looks a little bit like:
|
1 2 3 |
func Handler(ctx context.Context, w http.ResponseWriter, r *http.Request) { // Some functionality that may or may not need ctx } |
More importantly it’s not clear that ctx is associated with r and that the values within ctx or functionality shall affect the request or processing. For example:
|
1 2 3 4 |
func Handler(ctx context.Context, w http.ResponseWriter, r *http.Request) { e, found := entity.FromContext(ctx) // Processing... } |
vs
|
1 2 3 4 |
func Handler(w http.ResponseWriter, r *http.Request) { e, found := entity.FromContext(r.Context()) // Processing... } |
Attaching context to a request leads to code that is easier to understand and maintain. With that aside let’s have some fun with context, requests, and concurrency.
Note: There’s plenty of code left out to make the following fully functional. The code is meant to get an idea across rather than as a functional example or even a guideline.
When processing requests getting a response to the requestor quickly should be a high, if not the highest, priority. Many times middleware processing delays a response by serially processing headers or processing authentication tokens. The typical handler flow looks like:
The handler for step 2 may look like:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
func entityHandler(w http.ResponseWriter, r *http.Request) { c, err := r.Cookie("auth") if err != nil { // auth not found, let's go forward. // Don't worry about where next came from // I may write a post in the future about that. next(w, r) return } // Get the entity from some lookup system. This could take a long time. e := getEntity(c.Value) // Wrap around the context and send it onwards for processing. next(w, r.WithContext(context.WithValue(r.Context(), entityKey, e))) } |
Pretty straightforward and typical to almost all requests (you may also have CORS, rate limiting, etc.) and almost always run serially. Step 2 may involve database operations or network activity that blocks step 3 from parsing the request prior to reading values. We can potentially reduce the time to response by continuing processing while we’re fetching the entity:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
func entityHandler(w http.ResponseWriter, r *http.Request) { c, err := r.Cookie("auth") if err != nil { next(w, r) return } ed := entityData{} ed.w.Add(1) go func() { // Get the entity from some lookup system. This could take a long time. ed.e = getEntity(c.Value) ed.w.Done() }() // Wrap around the context and send it onwards for processing. next(w, r.WithContext(context.WithValue(r.Context(), entityKey, &ed))) } type entityData struct { w sync.WaitGroup e Entity } func FromContext(ctx context.Context) (Entity, bool) { ed, ok := ctx.Value(entityKey).(*entityData) if !ok { return Entity{}, false } // Block until processing is complete. ed.w.Wait() return ed.e, true } |
In the above we start a goroutine to get the entity while sending the request onwards for processing. If and when the entity value is needed FromContext shall block until the Entity has been retrieved; this may result in an immediate return if the Entity has already been returned. If the value is not needed then everything cleans itself up on the next garbage collector cycle.
The above is a toy example and an interesting thought. If you were to use a pattern like this then it’s worth investing in benchmarking and testing under realistic loads before you start down this route.
It’s been five years since my last blog post and while I had intended to post often at the time circumstances simply did not allow for it. Are things going to be different now? Possibly although there is even less free time. It’s more likely that I’ll start posting on specific activity blogs but I shall try to focus more here.
What’s happened in five years? Plenty of things to write about and I suspect that they’ll slowly leak out. In the meantime feel free to take a browse down memory lane (I have and some of things I’ve written are cringe worthy).
About a decade ago I started playing Counter Strike while in University. I started playing in the “Cove” or whatever the UW arcade was called at that time. I played CS, or at least started to, because I wanted to know what the buzz was all about. The first experience was interesting.
I was able to get into the game very quickly but also developed motion sickness almost immediately. Fighting through the sickness I managed to win a few rounds and was intrigued. I went on to purchase an upgraded machine on which I could play CS and spent many nights becoming a proficient player. Unfortunately I never made the jump to pro as I didn’t have the time to do so but was asked by several companies. I don’t think it was my skill that made me an attractive add but my teamwork skills and my ability to manage a team while I was also playing.
CS isn’t simply about taking out your opponents but about working as a unit in order to protect each other as well as overpower your enemy. If you see a unit isn’t throwing as many numbers at you as you should be seeing then it’s likely they’re flanking you and you need to counter it. The ability to see the full picture and react meant that I survived just long enough and took out just enough people to tilt the battle in favour of the better players on my team.
Of course, those mad skillz were put to use when one of the greatest challenges of my life came along:
It’s Bieber? I thought it was Beaver…
4chan is going to go nuts over this.
Regular visitors will notice the lack of a Movie or Media List. That may return someday. Things have changed substantially over the last few years and I’ll write an update post on that front soon.
You’ll also notice that the blog history now goes back to Sunday November 26th 2000 or my very first blog post…ever. You’ll also notice that I haven’t matured much since then or through the years. Talking about exceeding your goals!
For the third or fourth time in ten years a new platform is running this site.
While I toyed with using WordPress for a while I decided to spin my own (once again) as I have built a new web platform.
Let’s see where I go with this new system as it’s getting more user friendly by the day.
I know I haven’t posted in a year or so but this bit of information seemed important enough to share.
I was helping a family friend fix their 160GB iPod and Google searches proved fruitless so I thought I’d share the issue as well as how it was fixed.
Setup:
160GB iPod
iMac
Issue:
iPod appears in Finder (equivalent of Windows Explorer) but does not appear in iTunes.
Things tried that didn’t work:
Hard reset.
Disk mode.
Diagnostic mode.
Ejecting the iPod drive that appears in Finder causes iPod to disconnect and then instantly reset, almost as if it ran into a fatal error.
Eventual solution:
In Disk Utility I erased the drive partition and then ejected the iPod drive. The iPod then appeared in iTunes and worked normally including an update.
Possible issue:
The iPod drive appeared to be formatted to FAT and was using up the entire space available on the iPod. Ejecting the disc appeared to cause a problem as there was no space left for the iPod to actually store any media. Also to note, there was no content on the FAT partition; just empty space.
I’ve not traveled much by any stretch of the imagination. I have friends (like Mike Connolly) that have traveled around the world and experienced many cultures. My trips have been limited to the US, India, and parts of Europe. Yet it seems that I have more travel experience that most people. Why is it that people fail to travel anywhere except to pre-packaged resort all inclusive packages? Whatever happened to going on a road trip with only a compass and the only guides being the sun and the occasional advice from a stranger? Perhaps people are too busy to know better or want to package as much as possible into as little time as possible.
I try to take a distinctly opposing view by seeing things again and again. I like visiting some of the same places many times over and expanding the circle each time. That’s not to say that I don’t go to a new place but it’s impossible to see a whole town in one day. A town or city isn’t just historical sites; it’s the sidewalks, it’s the people or lack thereof in a park on any given day. Perhaps I take traveling too seriously.
Over the last few years my travel luggage has expanded from a single bag to four bags and now down to two. Experience teaches one that equipment can always be minimized. There are some things that just won’t be needed. In case of an emergency there’s very few things you worry about. For instance, last year when I went to Italy I crammed two camera bags full of equipment and necessities. This year I’m taking less than one camera bag and it has space for non-camera related items. It’s partially the experience of knowing what to shoot with and also deciding that I don’t really need 500 GB of RAID 1 backup on the go.
Do other people learn? Or perhaps other people are looking to see and not experience.