Image Surfer Pro Forms
Directed Search
The intended use of a Directed Search is to quickly process the Free Hosted Galleries linked to by a Thumbnail Post.
With this purpose in mind, Image Surfer Pro only crawls webpage links in a specific way and cannot be used as a general web spider to crawl the entire world wide web. At most a deep search will crawl three pages deep. While this seems to be a limitation, it keeps searches constrained to a reasonable processing time and efficiently finds the content for the intended use.
A Directed Search is done in two stages. First the pages are searched for content. Once all of the pages have been searched the content found is added to the fusker collection.
Choosing Specific Links For Directed Search
Next to each Page Link in the Links Column of the Image Table and Link Table there
is a check box to search the link for more data. When processed by the
button, a page search list will be created containing all of the selected page links.
A "Search All" box is available in the header of the Image Table and Link Table which can be used to quickly select groups of links to search from each table. When you want to search most of the links on a page, this gives you a quick starting point and you may then uncheck the boxes for the links you specifically don't want to search. These buttons will select only one of the three possible links on any given row. In order of priority Clean links are preferred followed by Sterile and finally Original.
NOTE: links which are direct references to images or videos will not have a Search Link check box. Simply use the Add Item check box to add the media link to your fusker collection.
Searching For Media Links
The search for media information builds several lists. The page search list originally created from the selected page links on the ISP Form is maintained and expanded during the process to assure pages are only searched once and to keep track of how far the page being searched is from the original page. The primary goal of the search is to find direct media file Links. Each type of media content is maintained in a separate list:
- Video List This list contains links to video files (such as MP4, FLV, WMV, etc.) as well as associated information such as the poster image URL, etc...
- Frame List This list contains information about active frame content found on the pages searched including the original size of the frame and a description of where the frame was found. It includes references to Shockwave Flash (SWF) files.
- Direct Images When an image is not directly visible on a page but is instead directly referenced by a hyper link on the page the image is typically of more interest for a fusker collection. This is the case with any FHG where it isn't the thumbnail image of interest but rather the image the thumbnail links to.
- Embedded Images In some cases the desired image content is never found in a Direct Image Link - instead it is embedded into the pages. The third list contains unique image information found embedded on some NOT ALL of the searched pages.
The way the lists are built and maintained depends on the "level" of the page being searched. The following describe how the search progresses through the page levels.
The First Page
Though you do not have to start from a Thumbnail Post, having a graphic to use in the example will help explain the process. Say each small image on the thumbnail post is a link to an Free Hosted Gallery. The Free Hosted Galleries are hosted by different domains.
|
Using both the
and
buttons:A Directed Search starts by creating an ISP Form. Image Surfer Pro requires this step only as a safety measure. Even clean and well trusted pages may have links you would not want to follow. For example links to book mark the page, use the page as your home page, etc. Creating an ISP Form first allows you to choose which links are searched by using the Search page for more data check box for each Free Hosted Gallery you wish to search. If the number of links you wish to search out numbers those you wish to ignore, using the "Search All" boxes at the top of the tables may make it easier. If you chose to add media links to your fusker collection by selecting the Add Item check boxes, those media references will be added to the collection prior to searching subsequent pages for additional content. |
Second Level Pages
Internet Explorer will navigate to each of the search links you selected in the ISP form created from the first page. It will navigate to each in the order they were listed in the form from top to bottom. Once the search page is loaded, Image Surfer Pro will process the page looking for media content. Image Surfer Pro processes these 2nd level pages in one of three ways based on the type of content and links it finds on the pages.
|
Direct Link Free Hosted Galleries: The most common type of FHG provides Direct Links to images or video files from thumbnail images on the page. Image Surer Pro will pull these direct links from the gallery page and add them to the appropriate lists. No new links are added to the Page Search List if sufficient video or image links are found to consider the page a direct link FHG.
Gallery Page Links:
Other Pages: |
As pages are searched all frame objects are kept, but since frames are often used to display active ads which would be of little interest to a collector, the number of frames found on a page is not used in determining how to treat the page.
Third Level Pages
While searching the second set of pages, Image Surfer Pro may have inserted additional pages into to the search list. This is done if it didn't find a sufficient number direct image or video links on the page but did find images which referenced other pages. Image Surfer Pro attempts to limit the number of these additional pages by not crossing domains. Only links which are found to be referenced by images and are in the same URL path as the referencing page will be added to the search list. In some cases this may cause Image Surfer Pro to miss some content you were expecting it to find but prevents a significant amount of wasted search time.
|
When processing this third level of webpages, Image Surfer Pro is expecting to find Media Pages.
Webpages where the primary content is a single image or video. All embedded media files on these
pages will be added to their specific list.
If the {Automatically collect embedded image links} setting on the User Preferences Images Tab Always every embedded image on every processed page is collected. In all other cases, only the embedded images from level three pages are added to the Embedded Images List. Image Surfer Pro will still add any Direct Image Links to the list of images found. Videos and Frames are also added to their lists when these pages are processed. No links will be added to the search list when processing these pages. |
What The Search Looks Like
As each page is searched it is displayed in the browser window. Thus the search also works as a slide show of the pages. You may configure a pause in the viewing of these pages in your user preferences on the Views Tab.
As the pages flip by an interactive Image Surfer Pro task progress window is shown at the top right corner of your primary screen. This progress window can be moved and you can continue to work in other applications or even other Internet Explorer windows while the search runs.
This progress window provides some very useful information, including what operation is being performed and which page it is being performed on. The total number of pages which have been searched along with the current count of total pages to search is provided along with the number of each media type found.
While the Elapsed Time counter is quite accurate, the Estimated Time Remaining may not be quite as accurate. The estimate is based upon the most recently processed pages and load time as well as processing time may vary greatly between pages. Since the number of pages to be searched may also grow the estimate of time remaining may suddenly become longer and the percentage of the progress bar completed may seem to move backwards at times during the search.
You may stop the search at any time by clicking the stop button on the status window. When you
do click the stop button it will change to a disabled "Stopping" button. The search will stop
after processing the next search page. When the search is stopped Image Surfer Pro will begin
adding the media content found prior to the stop command.
Once all of the pages have been searched for content, the Internet Explorer window will navigate back to the original Image Surfer Pro Form. All select boxes on the form will again be "unchecked".
Adding To The Fusker Collection
After the search for media content has completed or been stopped, the media content stored in the media lists will be added to the fusker collection. Each list is assemilated independently in order: Embedded Images, Directly Referenced Images, Videos, and Frames.
Videos, Frames, and Direct Images are always added into lists during a directed search regardless of the associated {Automatically collect...} configurations. These settings only determine whether or not each list is automatically assimilated.
During a Driected Search, how many embedded images will have been extracted and whether they are automatically assemilated depends on the {Automatically collect embedded images} configuration.
| Setting | Extracted | Assimilated |
|---|---|---|
| Always | From Every Page Processed | Always Automatic |
| Directed |
Pages where not many direct image or video references found. Typically directed search 3rd level pages or any directly processed webpage that isn't a FHG. |
Automatic only after a direct search |
| Never | Pages where not many direct image or video references found. Typically directed search 3rd level pages or any directly processed webpage that isn't a FHG. |
Never Automatic User Prompt if nothing automatically assimilated |
Because assimilation is dependent upon your User Preferences, it is possible to run a directed search and have no data automatically added to your fusker collection. When this happens, Image Surfer Pro will inform you what information was extracted in the search and let you chose which sets of data are added.
|
You may select as many of the different sets as you like when presented with the choice. Options where no
relevant data was extracted will be disabled.
Embedded images are always validated against the {Min embedded image file KBytes for auto collection} regardless of whether they are assemilated automatically or choice when nothing else was automatically assemilated. 
Because the original webpage was an ISP Form, the option to add the original webpage will always be grayed out after a directed search. In this context Video refers to MP4, FLV, and all windows media file references but does not include Shockwave flash files. Shockwave Flash references will be part of the Frames list. Some URLs without a recognized file extension may be included in either the Video or Frames list. MP4 declared sources for <video> tags will be in the video list while non specific sources for <iframe>, <embed>, and <object> tags will be in the frames list. |
Warning – If some of your data sets are set to automatically collect, those not set to automatically collect
will be discarded!
As each media list is added to the fusker collection an associated interactive progress window
will be shown in the top right corner of your primary screen. Videos, Frames, and Direct Images
are not validated in any way and will be added quickly. You may use the Stop button to skip some
or all of a specific media type without adversely affecting the addition of other media types.
Selecting stop will end the assimilation of that type of media and move onto the next type of
media.
Embedded images may or may not contain information you originally intended to find in your search.
It is not uncommon to have banner adds, headers, buttons, and assorted other graphics found by
a large search. In some cases data displayed as an image on the pages may not even link to a file
which could be directly accessed. To attempt to limit this clutter in your fusker collection, Image
Surfer Pro will compare the file size of each embedded image to your user preference. The
{Min image file size in Kbytes for auto collection add}
configuration on the Processing Tab allows you to tune how large embedded images need to be before
adding them to your fusker collections.
When All Is Done
Once all of the pages have been searched and all of the media references have been added to your fusker collection, this popup window will let you know the processing has completed and full browser control will be returned to you.
You will notice that most or all of the pages searched are not in "back history" of the browser. They do however exist in the browser history - if you wish to remove them from your browser history completely, you will need to manually clear your browser history.
