T-SQL Bitwise Operations

February 4, 2009 by Michelle Ufford
Filed under: Miscellaneous, Performance & Tuning, SQL Tips, Syndication 

I’ve seen bit-product columns from time-to-time, mostly in SQL Server 2000 system tables, but it’s never been something I’ve had to work with. And when I’ve needed to, I’ve known how to figure out which options are selected, i.e. a bit product of 9 means options 8 and 1 are selected. If you’ve ever taken a look at the [status] column on the sysdatabases table (SQL 2000), you’ll know what I’m talking about.

What I’ve never known how to do, until recently, was calculate these options programmatically. That’s why, when I noticed the [freq_interval] on the sysschedules table was a bit-product column, I decided to spend a little time figuring it out. Fortunately for me, a couple of my awesome co-workers, Jeff M. and Jason H., have worked with this before and were able to explain it to me. And, it turns out, it’s actually quite easy.

Let me back up a few steps in case you’re not familiar with this topic. If you check out the Books Online entry for the sysschedules table (2005), you’ll notice the following statement:

freq_interval is one or more of the following:
1 = Sunday
2 = Monday
4 = Tuesday
8 = Wednesday
16 = Thursday
32 = Friday
64 = Saturday

When I looked at the actual value in the table, the schedule has a [freq_interval] value of 42, which is the sum of the bit values for the days selected.

If there were more than 7 options, the bit values would continue to double, i.e. 128, 256, etc. And regardless of how many bit values you select, you’re guaranteed one and only one possible answer, as the sum of all previous bit values will never exceed the next bit value:
1 + 2 = 3
1 + 2 + 4 = 7
1 + 2 + 4 + 8 = 15

Knowing this, I’m able to retrieve the values manually: I start with the highest bit value that does not exceed 42, then subtract it; I repeat until I’m left with 0.

So…
42 – 32 = 10
10 – 8 = 2
2 – 2 = 0

That means my job is scheduled to run on Friday’s (32), Wednesday’s (8), and Monday’s (2).

Now how do I do this with T-SQL? SQL Server provides an operator specifically for this task: the bitwise AND operator (&). For now, I’m going to skip the “why” behind this and just get to the practical application. If you’re interested in the “why,” let me know and I’ll write a follow-up post on binary and logical AND and OR operations.

For example, to use the bitwise AND to find out which days are selected…

SELECT 42 & 1 AS 'Sunday'
    , 42 & 2 AS 'Monday'
    , 42 & 4 AS 'Tuesday'
    , 42 & 8 AS 'Wednesday'
    , 42 & 16 AS 'Thursday'
    , 42 & 32 AS 'Friday'
    , 42 & 64 AS 'Saturday';

… will return …

Sunday      Monday      Tuesday     Wednesday   Thursday    Friday      Saturday
----------- ----------- ----------- ----------- ----------- ----------- -----------
0           2           0           8           0           32          0

If the result is not equal to zero, then that day is selected. Easy as key lime pie, right?

Now let’s take it a step further and create our own working example. Let’s say we’re going to track the characteristics of various objects in a single bit-product column (note: this is not necessarily the best way to accomplish this in the real world, but it’s a good illustration). First, set up a table to use in our example. This table will have a column, [attributes], which will hold the sum of our bit values.

CREATE TABLE myTable
(
      id            INT IDENTITY(1,1)
    , item          VARCHAR(10)
    , attributes    INT
);
 
INSERT INTO myTable
SELECT 'Broccoli', 200 UNION All
SELECT 'Tomato', 193 UNION All
SELECT 'Car', 276 UNION All
SELECT 'Ball', 292;

Next, we’re going to create a table variable that holds characteristics and their values. We’ll then join these two tables together to see which attributes exist for each item.

DECLARE @statusLookup TABLE
(
      attribute INT
    , VALUE     VARCHAR(10)
);
 
INSERT INTO @statusLookup
SELECT 1, 'Red' UNION All
SELECT 4, 'Blue' UNION All
SELECT 8, 'Green' UNION All
SELECT 16, 'Metal' UNION All
 
SELECT 32, 'Plastic' UNION All
SELECT 64, 'Plant' UNION All
SELECT 128, 'Edible' UNION All
SELECT 256, 'Non-Edible';
 
SELECT a.item, b.VALUE
FROM myTable a
Cross Join @statusLookup b
WHERE a.attributes & b.attribute <> 0
ORDER BY a.item
    , b.VALUE

You should get this result:

item       value
---------- ----------
Ball       Blue
Ball       Non-Edible
Ball       Plastic
Broccoli   Edible
Broccoli   Green
Broccoli   Plant
Car        Blue
Car        Metal
Car        Non-Edible
Tomato     Edible
Tomato     Plant
Tomato     Red

Great, now we know broccoli is edible! Let’s apply a little XML to clean up the results…

SELECT a.item
    , REPLACE( REPLACE( REPLACE(( 
        SELECT VALUE 
        FROM @statusLookup AS b 
        WHERE a.attributes & b.attribute <> 0 
        ORDER BY b.VALUE FOR XML Raw)
        , '"/><row value="', ', '), '<row value="', ''), '"/>', '') 
        AS 'attributes'
FROM myTable a
ORDER BY a.item;
item       attributes
----------------------------------------
Ball       Blue, Non-Edible, Plastic
Broccoli   Edible, Green, Plant
Car        Blue, Metal, Non-Edible
Tomato     Edible, Plant, Red

Voila! There you have it, how to use the bitwise AND (&) operator to retrieve multiple values from a bit-product column. Pretty neat stuff!

Special thanks to Jeff M. and Jason H. for their assistance. :)

Happy Coding!

Michelle Ufford (aka SQLFool)

Source: http://sqlfool.com/2009/02/bitwise-operations/

Tags: , ,

Comments

13 Comments on T-SQL Bitwise Operations

  1. Colin Stasiuk on Wed, 4th Feb 2009 7:55 am

    2. Great article and examples… next time just remember “thinking blogs” should only be posted after noon LOL

  2. Margaret on Wed, 4th Feb 2009 12:20 pm

    3. Well, any 5 year old would argue with you about broccoli being edible and balls being blue, but there you go — what does a 5 year old know?

    Great example of using this function and quite cute as well!

    ê¿ê

  3. Denis Gobo on Thu, 5th Feb 2009 5:36 pm

    4. Michelle,nice post, see also my first post on sqlblog.com

    Speed Up Performance And Slash Your Table Size By 90% By Using Bitwise Logic
    http://sqlblog.com/blogs/denis_gobo/archive/2007/05/29/test.aspx

  4. Michelle Ufford on Fri, 6th Feb 2009 7:58 am

    5. Thanks, Denis, that’s great! The next task for me is to evaluate performance implications, and your article is giving me a jump start on that. :)

  5. View (and Disable) SQL Agent Jobs with TSQL : SQL Fool on Fri, 6th Feb 2009 8:07 am

    6. [...] Not sure what I’m doing with the @weekDay table? Then check out my post on bitwise operations in T-SQL. [...]

  6. Ranga on Thu, 16th Apr 2009 11:01 am

    7. Michelle,
    Just saw this website today…its great, like your style!

    I am kind of new to XML, couldn’t understand the & operator used in the below query: Can you please shed some light on this..

    WHERE a.attributes & b.attribute 0

    SELECT a.item
    , REPLACE( REPLACE( REPLACE((
    SELECT VALUE
    FROM #statusLookup AS b
    WHERE a.attributes & b.attribute 0
    ORDER BY b.VALUE FOR XML Raw)
    , ‘”/><row value=”‘, ‘, ‘), ”, ”)
    AS ‘attributes’
    FROM myTable a
    ORDER BY a.item;

  7. Michelle Ufford on Thu, 16th Apr 2009 11:42 am

    8. Hi Ranga,

    Thanks for the feedback! I’m glad you like the site. :)

    The & operator actually has nothing to do with XML; in the example above, XML was just used to help format the result set. The & operator is returning the bitwise value if the bitwise product contains it. To expand on the example above…

    CREATE TABLE #myTable
(
      myWeekDay                 VARCHAR(10)
    , bitwiseProduct            TINYINT
    , bitwiseValue              TINYINT
    , bitwiseOperatorResults    TINYINT
);
 
INSERT INTO #myTable
SELECT 'Sunday', 42, 1, 42 & 1 UNION All
SELECT 'Monday', 42, 2, 42 & 2 UNION All
SELECT 'Tuesday', 42, 4, 42 & 4 UNION All
SELECT 'Wednesday', 42, 8, 42 & 8 UNION All
SELECT 'Thursday', 42, 16, 42 & 16 UNION All
SELECT 'Friday', 42, 32, 42 & 32 UNION All
SELECT 'Saturday', 42, 64, 42 & 64;

    Now run these 2 queries to better examine what we’re doing with the WHERE a.attributes & b.attribute <> 0 syntax:

    SELECT * FROM #myTable;
 
SELECT *
FROM #myTable
WHERE bitwiseOperatorResults <> 0;

    The first query shows you the results of passing each of the values into the bitwise product, with the use of the & operator. The second query shows you only those values that are contained in the bitwise product, which means they are “selected.”

    Does that help? If not, let me know and I’ll expand on it further.

  8. Ranga on Mon, 20th Apr 2009 11:46 am

    9. Thanks a lot for the explanation….

  9. Another perl Blog » Blog Archive » The world is amazing: Perl: convert decimal to binary using … on Tue, 5th May 2009 7:33 am

    10. [...] Bitwise Operations [...]

  10. Index Defrag Script, v3.0 : SQL Fool on Tue, 23rd Jun 2009 5:37 pm

    11. [...] Take a SUM of the values for the days that you want excluded. So if you want an index to only be defragged on weekends, you would add up Monday through Friday (2+4+8+16+32) and use a value of 62 for the exclusionMask column. For a little more information on how this works, check out my blog post on Bitwise Operations. [...]

  11. Microsoft and DiscountASP.NET news on Wed, 24th Jun 2009 7:14 pm

    12. SQL Server Index Defrag Script…

    Hiya. . . Some of you may have already seen this, but just in case you haven’t; Some excellent SQL Server…

  12. SQL Notes « 36 Chambers – The Legendary Journeys: Execution to the max! on Sat, 10th Oct 2009 7:03 pm

    13. [...] after all.  Unfortunately, one of our legacy tables at work still uses these, so fortunately, Michelle Ufford has a good primer on how to do it.  Don’t do it, [...]

  13. Mike on Wed, 23rd Dec 2009 11:55 am

    14. This area of programming is becoming a lost art. Storing and retreiving data by using bit masks (as this article describes) is still a common use of bitwise operators, but there are so many other uses, namely data manipulation and calculation, that are becoming forgotton.

Tell me what you're thinking...
and oh, if you've enjoyed the post, please consider subscribing to my blog.