Image Surfer Pro Toolbar

Blow up of ISP Forms button from the Image Surfer Pro toolbar Image Surfer Pro Forms
Directed Search

The intended use of a Directed Search is to quickly process the Free Hosted Galleries linked to by a Thumbnail Post.

With this purpose in mind, Image Surfer Pro only crawls webpage links in a specific way and cannot be used as a general web spider to crawl the entire world wide web. At most a deep search will crawl three pages deep. While this seems to be a limitation, it keeps searches constrained to a reasonable processing time and efficiently finds the content for the intended use.

A Directed Search is done in two stages. First the pages are searched for content. Once all of the pages have been searched the content found is added to the fusker collection.

Choosing Specific Links For Directed Search

Next to each Page Link in the Links Column of the Image Table and Link Table there is a check box to search the link for more data. When processed by the Process Page button from Image Surfer Pro toolbar button, a page search list will be created containing all of the selected page links.

A "Search All" box is available in the header of the Image Table and Link Table which can be used to quickly select groups of links to search from each table. When you want to search most of the links on a page, this gives you a quick starting point and you may then uncheck the boxes for the links you specifically don't want to search. These buttons will select only one of the three possible links on any given row. In order of priority Clean links are preferred followed by Sterile and finally Original.

NOTE: links which are direct references to images or videos will not have a Search Link check box. Simply use the Add Item check box to add the media link to your fusker collection.

Searching For Media Links

The search for media information builds several lists. The page search list originally created from the selected page links on the ISP Form is maintained and expanded during the process to assure pages are only searched once and to keep track of how far the page being searched is from the original page. The primary goal of the search is to find direct media file Links. Each type of media content is maintained in a separate list:

The way the lists are built and maintained depends on the "level" of the page being searched. The following describe how the search progresses through the page levels.

The First Page

Though you do not have to start from a Thumbnail Post, having a graphic to use in the example will help explain the process. Say each small image on the thumbnail post is a link to an Free Hosted Gallery. The Free Hosted Galleries are hosted by different domains.

Graphical representation of a thumbnail post Using both the ISP Forms button from the Image Surfer Pro toolbar and Process Page button from the Image Surfer Pro toolbar buttons:
A Directed Search starts by creating an ISP Form. Image Surfer Pro requires this step only as a safety measure. Even clean and well trusted pages may have links you would not want to follow. For example links to book mark the page, use the page as your home page, etc. Creating an ISP Form first allows you to choose which links are searched by using the

Search page for more data

check box for each Free Hosted Gallery you wish to search. If the number of links you wish to search out numbers those you wish to ignore, using the "Search All" boxes at the top of the tables may make it easier.

If you chose to add media links to your fusker collection by selecting the Add Item check boxes, those media references will be added to the collection prior to searching subsequent pages for additional content.

Second Level Pages

Internet Explorer will navigate to each of the search links you selected in the ISP form created from the first page. It will navigate to each in the order they were listed in the form from top to bottom. Once the search page is loaded, Image Surfer Pro will process the page looking for media content. Image Surfer Pro processes these 2nd level pages in one of three ways based on the type of content and links it finds on the pages.

Graphical representation of an Image Gallery

Direct Link Free Hosted Galleries:
The most common type of FHG provides Direct Links to images or video files from thumbnail images on the page. Image Surer Pro will pull these direct links from the gallery page and add them to the appropriate lists. No new links are added to the Page Search List if sufficient video or image links are found to consider the page a direct link FHG.

Gallery Page Links:
Another common type of FHG links thumbnails to additional webpages where the desired media is embedded. If there are not a sufficient number of Direct media Links found, Image Surfer Pro will assume the content will be found on the 3rd level pages. Each link referenced by an image will then be inserted into the page search list.

Other Pages:
If Image Surfer Pro does not find sufficient direct media links on the page and does not find any new links to search referenced by images on the page, It will assume the desired content is actually on this page and will add any image found embedded on the page to a list of possible images of interest.

As pages are searched all frame objects are kept, but since frames are often used to display active ads which would be of little interest to a collector, the number of frames found on a page is not used in determining how to treat the page.

Third Level Pages

While searching the second set of pages, Image Surfer Pro may have inserted additional pages into to the search list. This is done if it didn't find a sufficient number direct image or video links on the page but did find images which referenced other pages. Image Surfer Pro attempts to limit the number of these additional pages by not crossing domains. Only links which are found to be referenced by images and are in the same URL path as the referencing page will be added to the search list. In some cases this may cause Image Surfer Pro to miss some content you were expecting it to find but prevents a significant amount of wasted search time.

Graphical representation of an Image Page

When processing this third level of webpages, Image Surfer Pro is expecting to find Media Pages. Webpages where the primary content is a single image or video. All embedded media files on these pages will be added to their specific list.

If the {Automatically collect embedded image links} setting on the User Preferences Images Tab Always every embedded image on every processed page is collected. In all other cases, only the embedded images from level three pages are added to the Embedded Images List.

Image Surfer Pro will still add any Direct Image Links to the list of images found. Videos and Frames are also added to their lists when these pages are processed.

No links will be added to the search list when processing these pages.

What The Search Looks Like

As each page is searched it is displayed in the browser window. Thus the search also works as a slide show of the pages. You may configure a pause in the viewing of these pages in your user preferences on the Views Tab.

As the pages flip by an interactive Image Surfer Pro task progress window is shown at the top right corner of your primary screen. This progress window can be moved and you can continue to work in other applications or even other Internet Explorer windows while the search runs.

Image Surfer Pro interactive progress window showing a Directed Search in progress

This progress window provides some very useful information, including what operation is being performed and which page it is being performed on. The total number of pages which have been searched along with the current count of total pages to search is provided along with the number of each media type found.

While the Elapsed Time counter is quite accurate, the Estimated Time Remaining may not be quite as accurate. The estimate is based upon the most recently processed pages and load time as well as processing time may vary greatly between pages. Since the number of pages to be searched may also grow the estimate of time remaining may suddenly become longer and the percentage of the progress bar completed may seem to move backwards at times during the search.

cut out of the stopping button for the interactive progress window You may stop the search at any time by clicking the stop button on the status window. When you do click the stop button it will change to a disabled "Stopping" button. The search will stop after processing the next search page. When the search is stopped Image Surfer Pro will begin adding the media content found prior to the stop command.

Once all of the pages have been searched for content, the Internet Explorer window will navigate back to the original Image Surfer Pro Form. All select boxes on the form will again be "unchecked".

Adding To The Fusker Collection

After the search for media content has completed or been stopped, the media content stored in the media lists will be added to the fusker collection. Each list is assemilated independently in order: Embedded Images, Directly Referenced Images, Videos, and Frames.

Videos, Frames, and Direct Images are always added into lists during a directed search regardless of the associated {Automatically collect...} configurations. These settings only determine whether or not each list is automatically assimilated.

During a Driected Search, how many embedded images will have been extracted and whether they are automatically assemilated depends on the {Automatically collect embedded images} configuration.

Detail of the portion of the User Configuration Images Tab dealing with the collection of Embedded Image urls
Setting Extracted Assimilated
Always From Every Page Processed Always Automatic
Directed Pages where not many direct image or video references found.
Typically directed search 3rd level pages or any directly processed webpage that isn't a FHG.
Automatic only after a direct search
Never Pages where not many direct image or video references found.
Typically directed search 3rd level pages or any directly processed webpage that isn't a FHG.
Never Automatic
User Prompt if nothing automatically assimilated

Because assimilation is dependent upon your User Preferences, it is possible to run a directed search and have no data automatically added to your fusker collection. When this happens, Image Surfer Pro will inform you what information was extracted in the search and let you chose which sets of data are added.

If no data was automatically added to the Fusker Collection this dialog allows the user to decide what extracted data they want addd to the fusker collection. You may select as many of the different sets as you like when presented with the choice. Options where no relevant data was extracted will be disabled.

Embedded images are always validated against the {Min embedded image file KBytes for auto collection} regardless of whether they are assemilated automatically or choice when nothing else was automatically assemilated.

Detail of the portion of the User Configuration Images Tab dealing with the size of Embedded Image urls

Because the original webpage was an ISP Form, the option to add the original webpage will always be grayed out after a directed search.

In this context Video refers to MP4, FLV, and all windows media file references but does not include Shockwave flash files. Shockwave Flash references will be part of the Frames list. Some URLs without a recognized file extension may be included in either the Video or Frames list. MP4 declared sources for <video> tags will be in the video list while non specific sources for <iframe>, <embed>, and <object> tags will be in the frames list.

Universal yellow triangle with ! Warning If some of your data sets are set to automatically collect, those not set to automatically collect will be discarded!

Screen capture of the Image Surfer Pro interactive progress window adding direct image links As each media list is added to the fusker collection an associated interactive progress window will be shown in the top right corner of your primary screen. Videos, Frames, and Direct Images are not validated in any way and will be added quickly. You may use the Stop button to skip some or all of a specific media type without adversely affecting the addition of other media types. Selecting stop will end the assimilation of that type of media and move onto the next type of media.


Screen capture of the Image Surfer Pro interactive progress window adding embedded image links Embedded images may or may not contain information you originally intended to find in your search. It is not uncommon to have banner adds, headers, buttons, and assorted other graphics found by a large search. In some cases data displayed as an image on the pages may not even link to a file which could be directly accessed. To attempt to limit this clutter in your fusker collection, Image Surfer Pro will compare the file size of each embedded image to your user preference. The {Min image file size in Kbytes for auto collection add} configuration on the Processing Tab allows you to tune how large embedded images need to be before adding them to your fusker collections.


When All Is Done

Pop up window when your Directed Search processing completes

Once all of the pages have been searched and all of the media references have been added to your fusker collection, this popup window will let you know the processing has completed and full browser control will be returned to you.

You will notice that most or all of the pages searched are not in "back history" of the browser. They do however exist in the browser history - if you wish to remove them from your browser history completely, you will need to manually clear your browser history.

Related User Preferences:

Image of User Preferences Dialog with the General tab selected - nothing highlighted Image of User Preferences Dialog with the Processing tab selected - Directed Search Configuration and Auto Optimize Configuration highlighted Image of User Preferences Dialog with the Processing tab selected - Auto select search options highlighted Image of User Preferences Dialog with the Views tab selected - nothing highlighted
Image of User Preferences Dialog with the Images tab selected - Collection and Optimization highlighted Image of User Preferences Dialog with the Videos tab selected - Collection and Optimization highlighted Image of User Preferences Dialog with the Frames tab selected - Collection and Optimization highlighted Image of User Preferences Dialog with the Pages tab selected - nothing highlighted


Processing Tab: Directed Search
The configurations available for Directed Search help you fine tune how the searches are performed and pages processed during the Directed Search. {Pause in seconds between search pages} allows you to customize the slide show effect of watching a Directed Search in progress. While {Page time out in Seconds} allows you to set the maximum time you wish to wait for a page to load. If the page does not finish loading in the configured time you will be asked if you wish to continue waiting or not.

Forms Tab: Choosing Links To Search
These two configurations correspond to the two "Search All" check boxes in the header of the Images Table and Link Table on the ISP Form. You can choose to automatically check one or both by default.

Images Tab: Image Collection
There are three controls for image collection. The first, {Automatically collect direct image links} deals with images that are not visible on the webpage but which were directly referenced by hyper-links on the processed pages. Direct image links will always be kept in the Direct Images List, but will not be added automatically to your fusker collection if this configuration is not checked.

The second two, {Automatically collect embedded image links} and {Min embedded image file Kbytes for auto collection} are applied to images visible on the processed pages. They determine if embedded images are automatically compared to the minimum file size required and added to the fusker collection. If {Automatically collect embedded image links} is set to Always or Directed Search, embedded images found during the Directed Search will be compared to the {Min embedded image file Kbytes for auto collection} size automatically. If it is set to Never the embedded images found will not be processed after the search.

Images Tab: Auto Optimize: If the {Auto combine individual Images into fusked images} configuration is checked, any images being added to the same directory in the fusker collection will be grouped into a fusked file. The form of the fusked file will be optimized and may be either a list or numeric fusk.

Videos Tab: Video Collection
The {Automatically collect video information found} configurations determines whether or not video file references found during the Directed Search are automatically added to the fusker collection.

Videos Tab: Auto Optimize
If the {Auto combine individual Videos into fusked Videos} configuration is checked, any videos being added to the same directory in the fusker collection will be grouped into a fusked file with other videos of the same type (e.g. FLV files group together but MP4 files do not group with FLV files). The form of the fusked file will be optimized and may be either a list or numeric fusk.

Frames Tab: Frame Collection
The {Automatically collect frame information found} configurations determines whether or not Shockwave Flash (SWF) file references and raw data found in <iframe>, <embed>, and <object> tags during the directed search is automatically added to the fusker collection.

Frames Tab: Auto Optimize
If the {Auto combine individual Frames into fusked Frames} configuration is checked, any frame information being added to the same directory in the fusker collection will be grouped into a fusked file with other frames of the same type (either SWF or Raw). The form of the fusked file will be optimized and may be either a list or numeric fusk.

Differences in Free and Full Versions

Screen capture of free version limitation dialog Directed Search:
The Free Version of Image Surfer Pro does not support processing Image Surfer Pro Forms. A Directed Search may only be started from an ISP Form and so is also not supported. This warning will be provided anytime the webpage to be processed is not a direct image link if you are using the free version of Image Surfer Pro.

Screen Capture Examples

Sample screen capture image after creating an ISP Form

The ISP Forms button and the Image Surfer Pro forms it creates can be used for many differnet purposes. The following examples will walk you through examples from the very basic to the more advanced.