{"id":6373,"date":"2014-10-12T07:22:12","date_gmt":"2014-10-12T11:22:12","guid":{"rendered":"http:\/\/esr.ibiblio.org\/?p=6373"},"modified":"2014-10-12T14:10:30","modified_gmt":"2014-10-12T18:10:30","slug":"a-low-performance-mystery-part-deux","status":"publish","type":"post","link":"http:\/\/esr.ibiblio.org\/?p=6373","title":{"rendered":"A low-performance mystery, part deux"},"content":{"rendered":"<p>Well, the good news is, I get to feel wizardly this morning.  Following sensible advice from a couple of my regulars, I rebuilt my dispatcher to use threads allocated at start time and looping until the list of masters is exhausted.<\/p>\n<p>78 LOC.  Fewer mutexes.  And it worked correctly <em>first time I ran it<\/em>. W00t &#8211; looks like I&#8217;ve got the <s>hang<\/s> non-hang of this threads thing.<\/p>\n<p>The bad news is, threaded performance is still atrocious in exactly the same way. Looks like thread-spawn overhead wasn&#8217;t a significant contributor.<\/p>\n<p>In truth, I was expecting this result.  I think my regulars were right to attribute this problem to cache- and locality-busting on every level from processor L1 down to the disks.  I believe I&#8217;m starting to get a feel for this problem from watching the performance variations over many runs.<\/p>\n<p>I&#8217;ll profile, but I&#8217;m sure I&#8217;m going to see cache misses go way up in the threaded version, and if I can find a way to meter the degree of disk thrashing I won&#8217;t be even a bit surprised to see that either.<\/p>\n<p>The bottom line here seems to be that if I want better threaded performance out of this puppy I&#8217;m going to have to at least reduce its working set a lot.  Trouble is, I&#8217;m highly doubtful &#8211; given what it has to do during delta assembly &#8211; that this is actually possible.  The CVS snapshots and deltas it has to snarf into memory to do the job are intrinsically both large and of unpredictably variable size.<\/p>\n<p>Maybe I&#8217;ll have an inspiration, but&#8230;Keith Packard, who originally wrote that code, is a damn fine systems hacker who is very aware of performance issues; if he couldn&#8217;t write it with a low footprint in the first place, I don&#8217;t judge my odds of second-guessing him successfully are very good.<\/p>\n<p>Ah well.  It&#8217;s been a learning experience. At least now I can say of multi-threaded application designs &#8220;Run! Flee! Save yourselves!&#8221; from a position of having demonstrated a bit of wizardry at them myself.<\/p>\n<p>UPDATE: One of my regulars found a minor bug in the mutex handling that cost some performance.  Alas, fixing this didn&#8217;t have any impact above the noise level of my profiling.  Also, I managed to unify the threaded and non-threaded dispatchers; the LOC specific to threading is now down to about 30.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Well, the good news is, I get to feel wizardly this morning. Following sensible advice from a couple of my regulars, I rebuilt my dispatcher to use threads allocated at start time and looping until the list of masters is exhausted. 78 LOC. Fewer mutexes. And it worked correctly first time I ran it. W00t&hellip; <a class=\"more-link\" href=\"http:\/\/esr.ibiblio.org\/?p=6373\">Continue reading <span class=\"screen-reader-text\">A low-performance mystery, part deux<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[13],"tags":[],"class_list":["post-6373","post","type-post","status-publish","format-standard","hentry","category-software","entry"],"_links":{"self":[{"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=\/wp\/v2\/posts\/6373","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=6373"}],"version-history":[{"count":4,"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=\/wp\/v2\/posts\/6373\/revisions"}],"predecessor-version":[{"id":6377,"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=\/wp\/v2\/posts\/6373\/revisions\/6377"}],"wp:attachment":[{"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=6373"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=6373"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/esr.ibiblio.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=6373"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}