ClearCase VOB size problem

[Apurva Shukla]

Hi all,
As far as I know, clearcase needs a VOB which cannot be partitioned in
any way.
What I mean by the above statement is: Suppose I set up a clearcase
setup on a server with 200GB capacity. Now work progresses and the VOB
size becomes 199 GB.
To continue working with ClearCase I might choose the following two
options.

1) Add further disk space
2) Free some disk space by partitioning the VOB into 2 pieces. I retain
the (present - n ) versions of all artifacts and rest I back it up on
the tape drive.

I am not very sure of the second option on clearcase and also dont know
any tool which has this capability. Kindly suggest if clearcase
supports this or if not then is there any other config management tool
which does?

Thanks in advance,
Apurva Shukla

[Marc Girod]

AS> As far as I know, clearcase needs a VOB which cannot be
AS> partitioned in any way.

ClearCase needs nothing. You do, and it depends on your purposes.
Vobs can be partioned in many ways. Again it depends on many things,
including what is meant by that.

AS> What I mean by the above statement is: Suppose I set up a
AS> clearcase setup on a server with 200GB capacity. Now work
AS> progresses and the VOB size becomes 199 GB.

There is not reason for you to use only one vob in the first place.
You can use 50 if you wish. Keep them small is a common and useful
advice.

Do you use UCM?
UCM is a well-know ClearCase killer. Remember you are not forced to
use it.

AS> I am not very sure of the second option on clearcase and also dont
AS> know any tool which has this capability.

Under base ClearCase (not UCM), the tool to split a vob is "ct
relocate". It will move your element history, do something sensible
with your types (copy them -- maybe share if your two vobs, old and
new, use a common admin vob; note that I didn't recommend using admin
vobs!).

It will break the config record of your derived objects, so nothing is
perfect.
Remember to clean up your lost+found directory before any such major
surgery.

[Phil Jarvis]

Post by Marc Girod
[...]
Do you use UCM?
UCM is a well-know ClearCase killer.

Why is that?

[Marc Girod]

Phil> Why is that?

Many reasons...
ClearCase was a tool without a policy.
It was aimed at letting *users* manage what they do.

UCM is a tool to enforce policies. It is for managers and
administrators to control that users do only what they (the
'mangers'/administraors) 'manage'.

In practice, with UCM, you have to decide first, and then apply
(typical example: an element *belongs* to a component, thus cannot be
relocated -- such decisions are made forever; an other example would
have been the extensive use of admin vobs -- same hierarchy
everywhere).

The focus is on source elements, with interest on derived objects
being dropped to almost insignificant:
- force on merging, thus invalidating existing DOs
- generated config specs based on intended (planned, controlled,...)
labels, instead on ones applied on top of DOs (with the -config
option), which is the bottom-up managed way.

<rant>
Of course, the driver behind UCM has been to conquer the Windows
market, with its careless users wanting GUIs to point&click, and not
interested in analysing what had actually happened: reboot and try
again.
</rant>

[Charles Jones]

You can also move the VOB storage onto a Storage Appliance type device.
Then you can expand disk space pretty much as much as you want.

[Apurva Shukla]

Thanks Marc for the description. I can see division of VOB in two ways.

1) Folder level division. Where I choose some folders to be in a
particular VOB
2) Version level division. Where I choose that version 1 to 20 stay in
VOB1 and 20 to latest stay in VOB2

MARC: Under base ClearCase (not UCM), the tool to split a vob is "ct
relocate". It will move your element history, do something sensible
with your types (copy them -- maybe share if your two vobs, old and
new, use a common admin vob; note that I didn't recommend using admin
vobs!).

I think you are talking about the 1st kind of division of VOBs. I feel
along with that 2nd kind of division is quite important and what I am
looking for. If after years of development (say 5 years) I plan to keep
relevant versions of the past two years only in all my VOBs and free up
the disk space by backing the rest on tape.
Is that also possible by ClearCase? and what commands if its possible.

TIA,
Apurva Shukla

[Marc Girod]

AS> I think you are talking about the 1st kind of division of VOBs.

Indeed.

AS> I feel along with that 2nd kind of division is quite important and
AS> what I am looking for. If after years of development (say 5 years)
AS> I plan to keep relevant versions of the past two years only in all
AS> my VOBs and free up the disk space by backing the rest on tape.

Well, I disagree.
Information that would not be linked would be of no value to me.

AS> Is that also possible by ClearCase? and what commands if its
AS> possible.

Well, sure: ct rmver (also rmbranch).
You can start with a

ct find . -ver '!created_since(1-Apr-2000)' -print

maybe grep away the '0' versions, and pipe the result to

xargs cleartool rmver

This will preserve "interesting" versions (with a warning), unless you
use -force.
Look at the -a option in order to adjust your config spec to access
what you might want to delete.

You might end up with a lot of empty elements, to rmelem, and stuff in
your lost+found. Maybe exclude directories, at least first...

But well, *I would not do that*
Be aware that you'll not be able to mount temporarily the backup
copies without first unregistering the current ones.

To reduce your vobs size, try first to:
- clean up lost+found
- scrub oplogs (and dump/load to reclaim the diskspace if you wish
so).
- split them with relocate as I told...

[Apurva Shukla]

Post by Marc Girod
Well, I disagree.
Information that would not be linked would be of no value to me.

Marc my concern about this kind of breakup is considering the size of
application and the data explosion which might take place in our
scenario of working. The space example of 200GB was just indicative.
Factually the application development is planned to run for about 10
years and might start with some 400GBs space requirement. And there is
a certainity that no one would want to go to a version beyond 3 years
old.

Your solution of rmver seems the best we can come near to solving this
problem. But here it completely removes the old data rather than
splitting the VOB. To plug this problem maybe we can have the whole set
of VOBs backed up and then do the deletion of old data (*uninteresting*
data)
Ideal scenario according to me could have been that clearcase splits
the VOB at versions and maintains the links between the splits. I back
up the older versions on DVDs or tapes. Now in case I happen to request
clearcase for some old version the clearcase just prompts me to put CD
number X to reconstruct that data for me.
So the data remains complete but in a form which does not put so much
load for the infrequently accessed data.
And thanks for the other suggestions to reduce size. Shall implement
them too.

Apurva Shukla

[Tim]

Post by Marc Girod
- clean up lost+found
- scrub oplogs (and dump/load to reclaim the diskspace if you wish
so).
- split them with relocate as I told...

Or use remote pools. The entire vob storage directory does not need
to be on one filesystem. You can move pools to other filesystems, or
add additional pools.

tim