Pruning Lines

(Creating a Master File) brought us to the foot of the mountain, now we start our ascent.
Here’s where you will have the opportunity to spend a tremendous amount of time pruning lines from your master file if you so desire.

While it’s tempting to try and be prepared for as much stuff as possible you have to draw the line somewhere. We are trying to get our books down to a size that we can use for review. You can always add a line to your book at a later date if you encounter something new from one of your opponents.

This is where reviewing your games is crucial. (A completely separate subject and one for a much later date.) Making mistakes and finding new ground are all part of the game. We just want to try and avoid making the same mistakes repeatedly.

If you have a good reference book(s) on openings you may want to keep it/them near by.
(MCO, ECO, Nunn’s, BCO, individual books on a particular opening, etc.)

For this exercise I’m going to use CB9’s “Reference Feature” to do the majority of my decision making on the lines to keep. If you don’t have CB9 you will have to search through your personal library and hope that you get all of main lines that your opponent can throw at you.

We are only going to cover the first set of Black’s possible responses after 1.e4 e5 2. Nf3
The possibilities were shown in the previous post as what to expect. Once you see how this is done you will see how easy it is to repeat the process for each of the branches.

Ok fire up the recently created Bookup file we called “E4-E5 Work”.
Crank up CB9 (Chessbase 9), open a new board.

Enter the following moves into CB9 1.e4 e5 2.Nf3 hit the “Reference tab” and wait a few moments while CB9 compiles a list of games. (It will take a fair amount of time in the early lines it gets noticeably faster the deeper into each line.)
This is basically a reprint of my earlier post no need to reinvent the wheel. :)

“My database contains 286,513 games

The report tells me that Black has played the following:
2...Nc6 (237,048) 83%*
2...Nf6 (31,153) 11%
2...d6 (14,147) 5%
2...f5 (1,904) .006%
2...d5 (938) .003%
2...Qe7 (533) .001
2...Bc5 (318) .001

* These are percentages that I have added to help throw some perspective on what to expect at this point. (Times occurred divided by total games found will give you the percentage) Example 2...Nc6 237,048 divided by 286,513 = .827 or 83% rounded up.

And even more moves than I have shown, but the number of times those moves have been seen in tournament play lessens significantly the farther we get from the top of the list. Does 2...Qg5?? Really need to be prepped?”

Ok now tab over to Bookup, it will probably be easier to resize the program windows so that you can see both the “CB9 board with reference tab info” and Bookup, unless you are really good at using Alt +Tab and remembering lots of info. But around my house it is nearly impossible. Those with children understand. :)

Now when you look at the starting position of the Bookup book/file “E4-E5 Work” we created you will see some lines for White that show possible transpositions.

Example: 1 d4 & 1 Nf3
You can delete everything but 1 e4. The same goes for everything found for Black’s replies except for 1...e5.
Now do the same with White’s second moves, delete everything but 2. Nf3 this way our
Opening book will match the move order of our current CB9 board.
Now switch your attention back to our CB9 board w/reference report.
As you can see this is a ton of information to digest and it is now time to start pruning heavily.
Obviously based on the frequency of occurrence we can expect to see the following
replies from Black.

2...Nc6
2...Nf6
2...d6
Just because
2...f5
2...e5
Haven’t occurred in a lot of GM games doesn’t mean we shouldn’t have something prepped. The Latvian and Elephant Gambits do happen quite often at the club level
and since this is our present fishbowl we might as well keep them in the mix.

So now I arrange and trim my “E4-E5 Work” file to match the report generated by CB9,
keeping just those 5 Black responses. I use the (Alt + U) to arrange the candidates in the same order as they appear in the CB9 reports. Make sure you set CB9 to prioritize the moves by number of times played. Simply click on the “Games” sub tab to make this happen.

Now in the CB9 “Ref Tab” selecting the top candidate (based on highest frequency) I click on 2...Nc6 and wait while it generates a new report.
It now gives me a long list of moves played by White.

I’m only interested in 3.Bc4 [The Italian or Guioco Piano (Pianissimo)], so I click on that move in the “Ref Tab”. While I’m waiting for a new report to be generated I toggle over to BU (Bookup “E4-E5 Work”) click on the move 2...Nc6, then I proceed to delete all of White’s 3^rd moves except 3.Bc4.

Once this is done I tab back over to CB9 and see what goodies Black has in store for us.
This is where it starts to get interesting. Black has a variety of very solid responses with a veritable who’s who of GM supporters for each one of the moves played.

We could spend hours at this point trying to decide what to keep and what to toss.
Fish or cut bait?
Keep all of the moves played above a certain ELO?
Pick a hypothetical number of lines?
Calculate the frequency played percentages?

All of this depends on you and how much time you are willing to invest.
I’m after main lines, since I’m using CB9 in this case as my primary guide. I pretty much have to look at each group of responses independently and truncate the lines once I establish a noticeable drop in frequency of occurrence. It’s either do it this way or spend a lot of time searching through other resources.

Like I said earlier you can always add lines at a later date if you face something new.
Our mission is to create an operational opening book that allows us to practice our new found knowledge. We aren’t trying to become a theoretical expert on any particular opening. We are striving for solid lines that allow us to direct matters into areas we understand.

For the sake of brevity I keep the top 4 responses for Black, and adjust my BU file accordingly.
3...Bc5
3...Nf6
3...Be7
3...d6

I now click on the move 3...Bc5 in CB and allow it to proceed with a new report, while waiting I make that same selection on my BU board.

White’s 4^th Move

Here is where knowing what you want to play saves you a lot of time.

Since I’m creating this book from White’s perspective I only want one move in my BU file in any of my openings. (Why add extra burden and additional study time, just having to worry about Black is enough work already.)

I know that I want to continue with 4.c3. So I click on that move in CB and while I’m waiting for the next report to finish I return to BU to trim away all moves except 4.c3 as White.

So far in this line we have played the following moves.

1 e4 e5
2 Nf3 Nc6
3 Bc4 Bc5
4 c3

Black’s 4^th Moves

Now there is a considerable drop in the diversity of responses from Black.
I’m prepping for the following:
4...Nf6
4...d7-d6

Continuing with my trend I update my BU file to match this move order, and select 4...Nf6 in CB to generate a new report.
In case you haven’t figured it out by now at some point we will have to return to
cover the other branches. By making these adjustments now I save time when I come back through here again.
It also makes it easier to take the highest move in the first pass, work through it until we hit a certain depth. Then back up one ply to solve each of the sub-variations.

White’s 5^th Move

5. d4
Once again I enter this move into CB9 first and trim lines out of BU while I wait for a new report. (It doesn’t take long for a new report to be generated once you get past move 3.)

I think you begin to get the idea, nothing fancy just a lot of work.

The main trick is to remember where you stopped, usually once I complete a line I will mark it in Bookup with a red, green, and yellow color code that I can later go back and remove under the commands option.

The other trick is to decide how far to take these lines. It all depends on your opponents
and the particular opening you are researching. You can always go further with your knowledge, because these opening books are something you can keep and modify.

Hope this has been helpful!
If you have any tips or suggestions please don't hesitate to add them here.

Creating a Master File

Okay enough procrastination on my part...

This is more along the lines of a how-to manual instead of my regular drivel. So I guess this should be filed under “How-to Drivel” or something equally impressive.

If someone happens to have a suggestion/short-cut/alternate route/or much needed improvement please don’t hesitate to post it here. The Knights are about feed-back and shared ideas. I promise I won’t take it personally.

Creating an opening book is really easy if you know what kind of positions you like to play. The hard part is doing enough research to have an understanding about which openings lead to said positions. But then again the best way to find out what you like is to play through the openings. You also have to decide whether or not you plan on playing the mainlines or do you plan to play systems in an attempt to reduce study time. There are pros and cons to both methods and I’m not even going to set foot in that territory.

Ok, with any cook book you need a list of ingredients:

(I’m using the following, but there are a variety of programs that could be substituted to achieve roughly the same results.)

Programs
Chessbase 9 (CB9)
Bookup 2000 Pro [Build 25] (BU)
Fritz 8 (F8)

Reference Material
Encyclopedia of Chess Openings A-E (ECO)

Reconnaissance and “Rough Draft”
As White I play 1. e4, of course after pushing my King’s pawn two squares forward
my opponent gets his/her say in the matter so I want to try and be prepared as much as I can.
Since I don’t know who my opponent is and what they might play I now start with the first of a seemingly long list of possibilities.
For this example I’m only worried about 1...e5
I meet 1...e5 with 2. Nf3. Using the ECO index I know that 1.e4,e5 falls under the C grouping. I’m not worried about possible transpositions at this point, I’m after information. So I fire up CB9, open a new board, and enter the previous moves then I hit the reference tab. This is when the fun begins, CB9 proceeds to scan the database that I have chosen to be used as my main source of information. (I’m using Megabase 2005, since it contains the most games and strongest players.) After a brief moment or two (depending on your computer speed) CB9 starts giving me a report of what I can expect to see as Black’s second move in order of frequency played.
The reason I use the “reference tab” as opposed to an “opening report” is because the “reference tab” will find transpositions and is faster for the needs of my rough draft.
If I want more detail I can do an “opening report”.
My database reports 286,513 games are found with this position. Notice I didn’t say move order because the position could in theory be reached by several move orders.
(1.Nf3, e5 2. e4), (1.Nf3,e6 2.e3,e5 3.e4), etc. [I just threw this tidbit in now because it is easier to explain transpositions with a simple example than trying to show something 6 moves deep. Now forget that I mentioned transpositions, and that we are looking for a list of Black’s second possibilities. :)]

The report tells me that Black has played the following:
2...Nc6 (237,048) 83%*
2...Nf6 (31,153) 11%
2...d6 (14,147) 5%
2...f5 (1,904) .006%
2...d5 (938) .003%
2...Qe7 (533) .001
2...Bc5 (318) .001

{* These are percentages that I have added to help throw some perspective on what to expect at this point.}

And even more moves than I have shown, but the number of times those moves have been seen in tournament play lessens significantly the farther we get from the top of the list. Does 2...Qg5?? Really need to be prepped?
Since I want to get through with this sometime before the turn of the next century
I need to establish some guidelines on as to how far I’m willing to prepare.
Where do I draw the line? That’s a tough one to answer, because the variations will continue to fluctuate along with the frequency. So it is at this point that I reach for my ECO for a little additional guidance. (I would probably skip the first step and just head straight to the ECO if it weren’t for the fact that some of the data in the ECO assumes you have the previous editions.)

Since the first five have the highest occurrence percentages I will start with them.
1.e4, e5 2.Nf3, Nc6 is shown as group C4. Turning to the C4 chapter gives
me a list of 10 diagrams that are numbered in succession C40-C49.
Here is where knowing what openings you want to play helps, and if you don’t know
this is a great place to gain some exposure.

C40 1.e4 e5 2.Nf3
C41 1.e4 e5 2.Nf3 d6
C42 1.e4 e5 2.Nf3 Nf6
C43 1.e4 e5 2.Nf3 Nf6 3.d4
C44 1.e4 e5 2.Nf3 Nc6
C45 1.e4 e5 2.Nf3 Nc6 3.d4 ed4 4.Nd4
C46 1.e4 e5 2.Nf3 Nc6 3.Nc3
C47 1.e4 e5 2.Nf3 Nc6 3.Nc3 Nf6
C48 1.e4 e5 2.Nf3 Nc6 3.Nc3 Nf6 4.Bb5
C49 1.e4 e5 2.Nf3 Nc6 3.Nc3 Nf6 4.Bb5 Bb4

The only codes that apply to me at the moment are C40, C41, C42
C40 covers all of the offbeat responses such as the Latvian, and Elephant Gambits.
C41 covers the Philidor lines.
C42 covers the Russian/Petroff Classical lines

The rest head into lines that I don’t happen to play at this moment, such as the Two Knights defense, and the Scotch.
All five of Black responses have been addressed except that I don’t see my response to 2...Nc6 in this group.
I meet 2...Nc6 with 3.Bc4. It must be in another set of ECO codes (C5 Group), so back to the book repeating the above process. Fortunately since I am trying to head into Guioco Piano
waters I get to eliminate ten tons of theory by avoiding the vast expanse known as the land of the Ruy Lopez. [Thank You Predrag! :)]
(I could also find each of the ECO codes using CB9 by selecting Tools/Opening Classification. I would have to scroll through each of the move orders, which would take some additional time. The advantage would be that is gives me the name of the general defense. There are plenty of on-line resources that list the moves with the related names
in great detail just do a search on “Openings classified by ECO code”. I usually just grab the code first and eventually the name.)
Now I know what ECO codes I can use to find or filter games.
Whether I create the files myself with CB9 or just download the games it really doesn’t matter. The main thing is that I want them in .pgn format because they are ultimately headed into a soon to be created Bookup file/book.
So now I gather all of the .pgn games I can find for each of the necessary ECO codes.

It’s not really important to have the most recent games or the strongest players for this part of the book building. All I’m really after is a lot of variations to create a master file for 1.e4 e5 2.Nf3 ... It is faster to prune lines out of an opening book than it is to add them. I learned this the hard way.

Now I fire up Bookup 2000 Pro and create a new Book called “E4-E5 Master”, you could call it anything you wish, it doesn’t really matter as long as you can keep track of it.

Select PGN/Import Games/ ->select the .pgn file. ->Reduce the number of plies to import
Down to 24 (12 moves deep, is plenty for my level of play.) De-select Highlight novelties->Click Ok and repeat the process for each of the needed .pgn files.

Once this accomplished there are just a few steps remaining before I can say that I’m finished with the Master file. Commands/Select “Clear Assessments”->Ok. This strips any numerical assessments from all of the positions in the book. (This may or may not be necessary, but I do it anyway.)

Now back to Commands/Select “Clear Rate Symbols”->Ok. This is necessary to clear the stray evaluations, and give you a clean slate. While doing the previous two steps you will have plenty of time to grab a beverage or a snack between commands.

You will want to back-up your newly created “Master” file. I just create a sub-folder for each master and copy the Bookup files into it. Just be sure that you get all of the files for each “Book” because Bookup uses a multitude a files.

After making a back-up Book of the “E4-E5 Master” as a safety check I would close out of all of the new “Books” and then reopen them one at a time to make sure they function properly. After they check out close the “Back-Up Master” and now rename the Book something like “E4-E5 Work”.

Finally... if you have managed to make it this far give yourself a pat on the back because that was a lot of work, and we have only reached the foot of the mountain.

We will start pruning lines in our next installment.