Understanding How Segments Merge
Every time new URL references are added to a fusker collection by any automated process fusker trees
are merged. This includes anytime the
button is used, any time the URL Capture Bar is used,
and anytime you choose Merge from the File Operations menu.
Merging starts at the top of the fusker tree with the Collection segment and works its way down each branch of the tree from top to bottom in order, pairing segments which fall at the same level on the same branch of the tree.
The merger process consists of 3 steps taken at each segment level:
- Decide if segments should be compared
- Match up the level of "Roll Up"
- Determine if segment information can be merged
- Absorb child segments
Decide If Segments Should Be Compared
In general segments are compared against other segments at the same location within the fusker tree. Segments from one branch are never compared against segments from another branch in the tree. However, during the merger of a fusker collection file an additional step is taken in deciding to compare segments. This step is not performed if the merger is due to processing a webpage or using the URL Capture Bar.
Disabling the {Remove duplicates from incoming collection} user configuration allows you to speed up merger processing by not comparing segments from a merging fusker collection file with other segments which came from that file. Before the file merger is begun, each segment within the fusker tree is marked as pre-existing in the tree. During the merger process if a segment branch does not match and is added to the collection directly from the incoming file, it will not have this mark. When a segment without this mark is asked to evaluate merger with an additional incoming segment, this new segment will evaluate the merger if this configuration is enabled and will immediately reject the merger prior to any other processing if the configuration is disabled.
This can significantly reduce the number of comparisons made if the fusker collection file contained a large fusker tree or directories with a large number of subdirectories or files. However, it means that any duplication of URL paths in the file will not be removed and will exist in the new collection (unless they duplicate something within the current fusker tree). In most cases, the change in merger speed may not be noticeable.
Whether of not you wish to enable this user configuration depends upon the types of fusker collection files you are merging and how they were constructed. There are multiple reasons the same file path may be duplicated in a fusker collection. Files may be duplicated simply to group the same media file with different sets of other files or two duplicate paths may also exist which contain very different file segments, etc. If you do not wish to preserve these existing duplicated structures you can enable this configuration and they will be merged together as the incoming file is processed.
Matching The Roll Up
Before we can determine if two segments can merge, we must first make sure they contain information about a similar portion of the fusker tree. To do this we need to match the number of directory branches contained in each segment. It may be necessary to Split or Roll Up in order to make an accurate comparison. This is done by counting the number of directory markers (i.e. "/") present in both segments then causing the new segment to either split or roll up the correct number of times to match the number of "/" markers.
The structure of the fusker tree being merged to the existing fusker tree is changed by this process. which makes the merger process Directional. If you merge two fusker collection files the result may be different based on which file is loaded first and which file is merged.
An example of the directionality of the merger process can be seen in how the following trees would merge.
| Generic Examples of matching the Roll Up level | ||
|---|---|---|
| In our first example our loaded tree does not have any split directories. The tree we wish to merge into our current tree has split the directory 3D_Model02 between 3D_ and Model02. Matching the Roll Up of the second tree to the first tree will roll up the split directory to create a directory segment 3D_Model02. Once this is done it is determined the directories do not match and a new branch of the tree is created. | ||
| Existing Tree | Merging Three | Resulting Tree |
 |
 |
 |
| If we reverse the order of the two collection files and merge the tree with no split directory into the tree which does have a split directory, we see a different result. In this case the directory 3D_Model01 is split in order to match the split directory 3D_. The remaining portion of the directory (Model01) doesn't match an existing branch of the split direcotry so is absorbed as a new branch below the split directory. | ||
| Existing Tree | Merging Three | Resulting Tree |
|
|
 |
This may seem quite complex, but in most cases you will find the results are actually quite intuitive. It may still be possible to create tree branches where the merger can't reconcile the roll up levels - in which case the two tree structures may both exist in some form within the merged tree even if a human could have made a better merger of the trees. In such cases you can manually manipulate the trees to have the structure you wish.
Forced Splitting
In general a Split Directory will have no directory markers unless it was manually manipulated after splitting. The result of matching the rollup of an incoming segment to a split directory can be one of three types of segments: Directory, File, or Split Directory. In the case of a URL from processing a webpage or the URL Capture Bar the result will always be either a directory or file segment.
The use of the {Force split segments based on string size} configuration can be used to modify the incoming segment after the rollup to force the incoming segment to be a split directory. The incoming segment will be split at the same number of characters present in the existing split segment. This splitting is essentially the same process that would have been used if the incoming directory matched the split directory, however since the directory is split before comparison, even if it does not match it will remain split.
|
Consider for example the fusker collection with a Split Directory segment 5281. When a new URL: http://www.studio.com/directory/models/290/previews/5221v1_1280x720.mp4 is merged into this collection the MP4 video file segment 5221v1_1280x720.mp4 will be compared to the split directory 5281. These segments are said to "Not Match" because the characters 5281 are not found at the beginning of the video text. |
|
If {Force split segments based on string size} is enabled, the video segment will be split after four characters (length of text in the split directory 5281} and the new collection will have two split directories (5281 and 5221) under the previews directory segment. |
|
However, if {Force split segments based on string size} is NOT enabled, the video segment will not be split, and the previews directory segment will now contain both the original split directory and the new video segment. |
The forced split will only occur if the incoming segment has enough characters to properly form the split. If the original split directory was only one of several segments at the same level and branch of the fusker tree, the rollup and force split functions are performed for each comparison made. If the incoming segment does not match any of the sibling segments, the final form will most closely match the last of the sibling segments.
While the {Force split segments based on string size} configuration can clearly be useful in some situations it may also split segments in undesirable ways if the URL paths at this point of the tree branch are not very uniform in construction.
Determining A Merger
While the general merge process is similar for each segment type, they all behave slightly differently and will combine with only segments of a similar type. If the information in the merging segment can not be combined with the existing segment the segment is passed back up to the parent for comparison with other branches or absorption. The following table describes which segment types will combine.
| Segment Type | Can Merge With | Condition | Result |
|---|---|---|---|
| Collection | Collection | Always | You choose which set of data to keep, the merging tree branches are absorbed |
| Path | Paths | Only if both text and protocol match | Exact matches combine the lower branches |
| Directory | Directories | One segment must contain completely and exactly the other segment's information. |
The most complete set of information is retained and the child segments of the merging tree are
absorbed. If one or both of the segments are fusked, the larger fusk is maintained The directory or split directory which matches the most characters of the merging segment will be chosen as the "best" match. This allows identical matching regardless of the order of the segments in the current fusker collection. |
| Split Directory |
Split Dir Directories Files |
The information in the existing segment must be completely contained in the merging segment. There may also be additional information in the merging segment which can be used to form a child segment. |
The most complete set of information is retained. If one or both segments are fusked, the larger
fusk is retained. Any additional information contained in the merging segment will be used to
form a child segment for absorption. The directory or split directory which matches the most characters of the merging segment will be chosen as the "best" match. This allows identical matching regardless of the order of the segments in the current fusker collection. |
| File | Files of same type | File Segments combine only with other files of the same type (Images with Images etc.). Files combine based on the {Auto combine individual files into fusked files} User Configuration for their file type. | If this configuration is enabled, individual file segments will be combined into an optimized fusked file segment. Otherwise individual files will be listed individually under the same directory or split directory segment. |
When file segments are merged because the {Auto combine individual files into fusked files} configuration is enabled, the entire text (both filename and extension) of each file referenced by both segments is first formed into a single list fusk. During the formation of the list fusk, duplicates are removed and what remains is a list of each file at the destination path referenced by both of the original file segments. The Optimization Process is then applied to the newly created fusked segment.
There are four {Auto combine individual files into fusked files} configurations in the User Preferences. Each applies to a specific group of segment types:
| Configuration | Applies To |
|---|---|
| {Auto combine individual Images into fusked Images} |
|
| {Auto combine individual Videos into fusked Videos} |
|
| {Auto combine individual Frames into fusked Frames} |
|
| {Auto combine individual Pages files into fusked Pages} |
|
When file segments with Poster Image or Text Description fields merge, the resulting segment will have only one of the two fields. If the existing segment already had the field, that text will remain. If not, the text (if any) of the merging segment will be used.
When file segments with Segment Sizing information merge, the sizing information from the segment already in the fusker collection will be used in all cases.
Absorbing Child Branches
The process of absorbing the child branches of the merging segment is an iterative and recursive process. This means we will ask each of our child branches if it could merge with each of the child branches of the segment we merged with. Each child responds with how well it could merge with each of the incoming children and the "Best" matches are made. Once one of our child segments is combined with the information from the merging segment's child, the process will continue down the tree as the next level of child segments are matched up.
Each of our child branches will be given the opportunity to combine with a child branch of the merging segment. The quality of matches is determined by the number of matching characters such that the order of the segments in each tree will not matter unless there are duplicate segments in the existing fusker collection - in which case the first of the duplicates will always be chosen.
Things to Consider
In most cases merging will lead to the expected results, but there are a few things that can lead to rather confusing results. While each of the following items are all valid and the results of their merging quite deterministic, the combination of the two may not be as intuitive as for more standard practices. In general the following situations can only occur through manual manipulation of the fusker tree. As they are manual changes to segments, they by definition are attempting to exploit patterns in the directory structure of servers. These manual manipulations may sometimes cause situations where merging additional files into the tree may not produce the desired result. In order to accurately add additional files into such manually generated fusking constructs may require additional manual manipulations.
