{"id":187,"date":"2012-09-05T15:12:27","date_gmt":"2012-09-05T20:12:27","guid":{"rendered":"http:\/\/thenoyes.com\/littlenoise\/?p=187"},"modified":"2012-09-05T15:12:27","modified_gmt":"2012-09-05T20:12:27","slug":"auto_increment-gaps","status":"publish","type":"post","link":"https:\/\/thenoyes.com\/littlenoise\/?p=187","title":{"rendered":"Auto_increment gaps"},"content":{"rendered":"<p>Why are there gaps in my auto_increment sequence, even if there are no deletes or rolled back transactions?<\/p>\n<p>Is it a <a href=\"http:\/\/bugs.mysql.com\/bug.php?id=34696\" title=\"bug\">bug<\/a>?<\/p>\n<p>The <a href=\"http:\/\/dev.mysql.com\/doc\/refman\/5.5\/en\/innodb-auto-increment-handling.html#innodb-auto-increment-configurable\" title=\"manual\">manual<\/a> says, &#8220;For lock modes 1 or 2, gaps may occur between successive statements because for bulk inserts the exact number of auto-increment values required by each statement may not be known and overestimation is possible.&#8221;<\/p>\n<p>Where does that overestimation come from?<\/p>\n<p>An example to illustrate:<\/p>\n<p><code>DROP TABLE IF EXISTS t;<br \/>\nCREATE TABLE t (a bigint unsigned auto_increment primary key) ENGINE=InnoDB SELECT NULL AS a;<br \/>\n\/* #1 *\/ INSERT INTO t SELECT NULL FROM t;<br \/>\n\/* #2 *\/ INSERT INTO t SELECT NULL FROM t;<br \/>\n\/* #3 *\/ INSERT INTO t SELECT NULL FROM t;<br \/>\n\/* #4 *\/ INSERT INTO t SELECT NULL FROM t;<br \/>\nSELECT * FROM t;<\/p>\n<p>+----+<br \/>\n| a  |<br \/>\n+----+<br \/>\n|  1 |<br \/>\n|  2 |<br \/>\n|  3 |<br \/>\n|  4 |<br \/>\n|  6 |<br \/>\n|  7 |<br \/>\n|  8 |<br \/>\n|  9 |<br \/>\n| 13 |<br \/>\n| 14 |<br \/>\n| 15 |<br \/>\n| 16 |<br \/>\n| 17 |<br \/>\n| 18 |<br \/>\n| 19 |<br \/>\n| 20 |<br \/>\n+----+<br \/>\n16 rows in set (0.02 sec)<\/code><\/p>\n<p>Notice that 5 and 10-12 are missing. If we did another insert, we&#8217;d be missing 21-27 (try it and see!)<\/p>\n<p>Here&#8217;s a model of what MySQL is doing:<\/p>\n<p>Create the table and simultaneously insert a single row. That is the row where a=1.<\/p>\n<p>#1: Insert as many rows as there are in the table (it&#8217;s one row, but MySQL doesn&#8217;t know that.)<br \/>\n&#8211; Grab a chunk of auto_increment values. How many in the chunk? One &#8211; the value &#8216;2&#8217;. Insert it (one row inserted).<br \/>\n&#8211; No more rows to insert, so all done.<\/p>\n<p>#2: Insert as many rows as there are in the table (it&#8217;s two rows, but MySQL doesn&#8217;t know that.)<br \/>\n&#8211; Grab a chunk of auto_increment values. How many in the chunk? One &#8211; the value &#8216;3&#8217;. Insert it (one row inserted).<br \/>\n&#8211; Still more rows to insert. Grab another chunk, twice as big as before &#8211; two values, &#8216;4&#8217; and &#8216;5&#8217;. Insert the &#8216;4&#8217; (two rows inserted).<br \/>\n&#8211; No more rows to insert. Discard the left over &#8216;5&#8217;.<\/p>\n<p>#3: Insert as many rows as there are in the table (it&#8217;s four rows, but MySQL doesn&#8217;t know that.)<br \/>\n&#8211; Grab a chunk of auto_increment values. How many in the chunk? One &#8211; the value &#8216;6&#8217;. Insert it (one row inserted).<br \/>\n&#8211; Still more rows to insert. Grab another chunk, twice as big as before &#8211; two values, &#8216;7&#8217; and &#8216;8&#8217;. Insert them (three rows inserted).<br \/>\n&#8211; Still more rows to insert. Grab another chunk, twice as big as before &#8211; four values, &#8216;9&#8217;, &#8217;10&#8217;, &#8217;11&#8217;, &#8217;12&#8217;. Insert the &#8216;9&#8217; (four rows inserted).<br \/>\n&#8211; No more rows to insert. Discard the left over &#8217;10&#8217;, &#8217;11&#8217;, and &#8217;12&#8217;.<\/p>\n<p>#4: Insert as many rows as there are in the table (it&#8217;s eight rows, but MySQL doesn&#8217;t know that.)<br \/>\n&#8211; Grab a chunk of auto_increment values. How many in the chunk? One &#8211; the value &#8217;13&#8217;. Insert it (one row inserted).<br \/>\n&#8211; Still more rows to insert. Grab another chunk, twice as big as before &#8211; two values, &#8217;14&#8217; and &#8217;15&#8217;. Insert them (three rows inserted).<br \/>\n&#8211; Still more rows to insert. Grab another chunk, twice as big as before &#8211; four values, &#8217;16&#8217;, &#8217;17&#8217;, &#8217;18&#8217;, &#8217;19&#8217;. Insert them (seven rows inserted).<br \/>\n&#8211; Still more rows to insert. Grab another chunk, twice as big as before &#8211; eight values, &#8217;20&#8217;, &#8217;21&#8217;, &#8217;22&#8217;, &#8230;, &#8217;27&#8217;. Insert the &#8217;20&#8217; (eight rows inserted).<br \/>\n&#8211; No more rows to insert. Discard the left over &#8217;21&#8217;, &#8217;22&#8217;, etc.<\/p>\n<p>The gap can get as big as 65535 (I didn&#8217;t look in the code to confirm that, it&#8217;s just what running the test above a few more times seems to suggest).<\/p>\n<p>When innodb_autoinc_lock_mode=1, there can be gaps between statements, but not within a statement, because there&#8217;s a lock on the auto_increment until we&#8217;re done. #4 above is guaranteed to get 8 consecutive values, you just might not be sure where they are going to start (well, now you are, because you read this post).<\/p>\n<p>When innodb_autoinc_lock_mode=2, there can be gaps within a statement, because the auto_increment is not locked. Imagine we&#8217;re in the middle of #4 above. Our statement is inserting the &#8217;14&#8217; and &#8217;15&#8217;, when another statement comes along wanting just a single auto_increment value. It gets the &#8217;16&#8217;. Now it&#8217;s our turn to ask for another chunk, and we get &#8217;17&#8217;, &#8217;18&#8217;, &#8217;19&#8217;, &#8217;20&#8217;. While we&#8217;re doing those, another statement comes along and steals our &#8217;21&#8217;. So the last row for our statement is &#8217;22&#8217;.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Why are there gaps in my auto_increment sequence, even if there are no deletes or rolled back transactions? Is it a bug? The manual says, &#8220;For lock modes 1 or 2, gaps may occur between successive statements because for bulk inserts the exact number of auto-increment values required by each statement may not be known [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[4],"tags":[],"class_list":["post-187","post","type-post","status-publish","format-standard","hentry","category-mysql"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p2IBF1-31","jetpack_sharing_enabled":true,"jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/thenoyes.com\/littlenoise\/index.php?rest_route=\/wp\/v2\/posts\/187","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/thenoyes.com\/littlenoise\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/thenoyes.com\/littlenoise\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/thenoyes.com\/littlenoise\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/thenoyes.com\/littlenoise\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=187"}],"version-history":[{"count":4,"href":"https:\/\/thenoyes.com\/littlenoise\/index.php?rest_route=\/wp\/v2\/posts\/187\/revisions"}],"predecessor-version":[{"id":191,"href":"https:\/\/thenoyes.com\/littlenoise\/index.php?rest_route=\/wp\/v2\/posts\/187\/revisions\/191"}],"wp:attachment":[{"href":"https:\/\/thenoyes.com\/littlenoise\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=187"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/thenoyes.com\/littlenoise\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=187"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/thenoyes.com\/littlenoise\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=187"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}