1 <codewalk title="Share Memory By Communicating"> 2 3 <step title="Introduction" src="doc/codewalk/urlpoll.go"> 4 Go's approach to concurrency differs from the traditional use of 5 threads and shared memory. Philosophically, it can be summarized: 6 <br/><br/> 7 <i>Don't communicate by sharing memory; share memory by communicating.</i> 8 <br/><br/> 9 Channels allow you to pass references to data structures between goroutines. 10 If you consider this as passing around ownership of the data (the ability to 11 read and write it), they become a powerful and expressive synchronization 12 mechanism. 13 <br/><br/> 14 In this codewalk we will look at a simple program that polls a list of 15 URLs, checking their HTTP response codes and periodically printing their state. 16 </step> 17 18 <step title="State type" src="doc/codewalk/urlpoll.go:/State/,/}/"> 19 The State type represents the state of a URL. 20 <br/><br/> 21 The Pollers send State values to the StateMonitor, 22 which maintains a map of the current state of each URL. 23 </step> 24 25 <step title="Resource type" src="doc/codewalk/urlpoll.go:/Resource/,/}/"> 26 A Resource represents the state of a URL to be polled: the URL itself 27 and the number of errors encountered since the last successful poll. 28 <br/><br/> 29 When the program starts, it allocates one Resource for each URL. 30 The main goroutine and the Poller goroutines send the Resources to 31 each other on channels. 32 </step> 33 34 <step title="Poller function" src="doc/codewalk/urlpoll.go:/func Poller/,/\n}/"> 35 Each Poller receives Resource pointers from an input channel. 36 In this program, the convention is that sending a Resource pointer on 37 a channel passes ownership of the underlying data from the sender 38 to the receiver. Because of this convention, we know that 39 no two goroutines will access this Resource at the same time. 40 This means we don't have to worry about locking to prevent concurrent 41 access to these data structures. 42 <br/><br/> 43 The Poller processes the Resource by calling its Poll method. 44 <br/><br/> 45 It sends a State value to the status channel, to inform the StateMonitor 46 of the result of the Poll. 47 <br/><br/> 48 Finally, it sends the Resource pointer to the out channel. This can be 49 interpreted as the Poller saying "I'm done with this Resource" and 50 returning ownership of it to the main goroutine. 51 <br/><br/> 52 Several goroutines run Pollers, processing Resources in parallel. 53 </step> 54 55 <step title="The Poll method" src="doc/codewalk/urlpoll.go:/Poll executes/,/\n}/"> 56 The Poll method (of the Resource type) performs an HTTP HEAD request 57 for the Resource's URL and returns the HTTP response's status code. 58 If an error occurs, Poll logs the message to standard error and returns the 59 error string instead. 60 </step> 61 62 <step title="main function" src="doc/codewalk/urlpoll.go:/func main/,/\n}/"> 63 The main function starts the Poller and StateMonitor goroutines 64 and then loops passing completed Resources back to the pending 65 channel after appropriate delays. 66 </step> 67 68 <step title="Creating channels" src="doc/codewalk/urlpoll.go:/Create our/,/complete/"> 69 First, main makes two channels of *Resource, pending and complete. 70 <br/><br/> 71 Inside main, a new goroutine sends one Resource per URL to pending 72 and the main goroutine receives completed Resources from complete. 73 <br/><br/> 74 The pending and complete channels are passed to each of the Poller 75 goroutines, within which they are known as in and out. 76 </step> 77 78 <step title="Initializing StateMonitor" src="doc/codewalk/urlpoll.go:/Launch the StateMonitor/,/statusInterval/"> 79 StateMonitor will initialize and launch a goroutine that stores the state 80 of each Resource. We will look at this function in detail later. 81 <br/><br/> 82 For now, the important thing to note is that it returns a channel of State, 83 which is saved as status and passed to the Poller goroutines. 84 </step> 85 86 <step title="Launching Poller goroutines" src="doc/codewalk/urlpoll.go:/Launch some Poller/,/}/"> 87 Now that it has the necessary channels, main launches a number of 88 Poller goroutines, passing the channels as arguments. 89 The channels provide the means of communication between the main, Poller, and 90 StateMonitor goroutines. 91 </step> 92 93 <step title="Send Resources to pending" src="doc/codewalk/urlpoll.go:/Send some Resources/,/}\(\)/"> 94 To add the initial work to the system, main starts a new goroutine 95 that allocates and sends one Resource per URL to pending. 96 <br/><br/> 97 The new goroutine is necessary because unbuffered channel sends and 98 receives are synchronous. That means these channel sends will block until 99 the Pollers are ready to read from pending. 100 <br/><br/> 101 Were these sends performed in the main goroutine with fewer Pollers than 102 channel sends, the program would reach a deadlock situation, because 103 main would not yet be receiving from complete. 104 <br/><br/> 105 Exercise for the reader: modify this part of the program to read a list of 106 URLs from a file. (You may want to move this goroutine into its own 107 named function.) 108 </step> 109 110 <step title="Main Event Loop" src="doc/codewalk/urlpoll.go:/range complete/,/\n }/"> 111 When a Poller is done with a Resource, it sends it on the complete channel. 112 This loop receives those Resource pointers from complete. 113 For each received Resource, it starts a new goroutine calling 114 the Resource's Sleep method. Using a new goroutine for each 115 ensures that the sleeps can happen in parallel. 116 <br/><br/> 117 Note that any single Resource pointer may only be sent on either pending or 118 complete at any one time. This ensures that a Resource is either being 119 handled by a Poller goroutine or sleeping, but never both simultaneously. 120 In this way, we share our Resource data by communicating. 121 </step> 122 123 <step title="The Sleep method" src="doc/codewalk/urlpoll.go:/Sleep/,/\n}/"> 124 Sleep calls time.Sleep to pause before sending the Resource to done. 125 The pause will either be of a fixed length (pollInterval) plus an 126 additional delay proportional to the number of sequential errors (r.errCount). 127 <br/><br/> 128 This is an example of a typical Go idiom: a function intended to run inside 129 a goroutine takes a channel, upon which it sends its return value 130 (or other indication of completed state). 131 </step> 132 133 <step title="StateMonitor" src="doc/codewalk/urlpoll.go:/StateMonitor/,/\n}/"> 134 The StateMonitor receives State values on a channel and periodically 135 outputs the state of all Resources being polled by the program. 136 </step> 137 138 <step title="The updates channel" src="doc/codewalk/urlpoll.go:/updates :=/"> 139 The variable updates is a channel of State, on which the Poller goroutines 140 send State values. 141 <br/><br/> 142 This channel is returned by the function. 143 </step> 144 145 <step title="The urlStatus map" src="doc/codewalk/urlpoll.go:/urlStatus/"> 146 The variable urlStatus is a map of URLs to their most recent status. 147 </step> 148 149 <step title="The Ticker object" src="doc/codewalk/urlpoll.go:/ticker/"> 150 A time.Ticker is an object that repeatedly sends a value on a channel at a 151 specified interval. 152 <br/><br/> 153 In this case, ticker triggers the printing of the current state to 154 standard output every updateInterval nanoseconds. 155 </step> 156 157 <step title="The StateMonitor goroutine" src="doc/codewalk/urlpoll.go:/go func/,/}\(\)/"> 158 StateMonitor will loop forever, selecting on two channels: 159 ticker.C and update. The select statement blocks until one of its 160 communications is ready to proceed. 161 <br/><br/> 162 When StateMonitor receives a tick from ticker.C, it calls logState to 163 print the current state. When it receives a State update from updates, 164 it records the new status in the urlStatus map. 165 <br/><br/> 166 Notice that this goroutine owns the urlStatus data structure, 167 ensuring that it can only be accessed sequentially. 168 This prevents memory corruption issues that might arise from parallel reads 169 and/or writes to a shared map. 170 </step> 171 172 <step title="Conclusion" src="doc/codewalk/urlpoll.go"> 173 In this codewalk we have explored a simple example of using Go's concurrency 174 primitives to share memory through communication. 175 <br/><br/> 176 This should provide a starting point from which to explore the ways in which 177 goroutines and channels can be used to write expressive and concise concurrent 178 programs. 179 </step> 180 181 </codewalk> 182
