Extreme Design Blog

Software solutions for complex problems

Getting started with a custom screen-scraper project

This document contains a a general overview and links to get you started with your custom screen-scraper project.  Some of the topics may not apply to your project.

  • Getting screen-scraper – If you do not have screen-scraper, please visit: http://www.screen-scraper.com/download/?l=p
  • Importing your project – To import your project (.sss) file, see Importing a screen-scraper project
  • Importing XML data into Excel – Many project export the data to a generic XML format, to import data data into Excel, see Import XML into Excel
  • Import XML into Excel

    Note: this document was written for Excel 2010.

    1. The application will output the custom XML to one of the following locations:
      1. C:\Scrapes\<Project Name>\<Project Name>.xml
      2. <Screen-scraper installation>\<Project Name>.xml
    2. Definition
      1. <Project Name>- this is name of the project
      2. <Screen-scraper installation> – this is the installation location of screen-scraper
    3. From Excel,  select:
      1. Data 
      2. Get External Data 
      3. From Other Sources 
      4. From XML Data Import
    4. In the file open dialog, select the xml file to import (see step 1 for the file name and path).
    5. If a cell location dialog open, click ok to insert the data.

    NOTE: you can also drag the xml file directly into Excel, dropping the file into Excel will start the import process.  Click OK (accepting the default settings) to import the data.

    Importing a screen-scraper project

    If you have not installed screen-scraper, please visit screen-scraper.com before continuing with this topic.

    1. Open Screen Scraper
    2. Select File > Import…
    3. NOTE: if the file you are about to import already exists (for example you are importing an update to an existing scrape) then you first need to delete the current scrape  
    4. Select the file to import (found in the installation folder of your custom project)
    5. Select the Scraping Session to run
    6. Click Run Scraping Session
    7. �����Â

    8. NOTE: for best performance set the log output to 0 (zero)
      1. In the scraping session
      2. Select the Log tab
      3. Set the log value to 0 (zero)

    How do you know if you can use Business Automation?

    Sometimes the biggest hurdle to automating system is understanding what systems can be automated.  When referring to automating applications such as Excel or Word, we often here, “I didn’t know you could do that!”

    With today’s technology, most systems offer support for some sort of automation.  Many of them have a large feature set designed just for automation. Let’s use Microsoft Word as an example.  It’s a great application for word processing.   Over the years it has grown to include a large number of additional features such as a mail merge, image formating, basic web design and so on.  Even with the additional of new features one major issue still exists, the product is designed to cover many possible usage types.  Covering all the bases is great if you have many different task to accomplish with the same application (Word in this case).  In covering many possible requirements, a product will naturally have limited steps specific to your company’s needs.  Typically this is by design.  After all, how do you make a software product for several million users?

    This is were business automation takes over!  One definition of business automation is: Applying company or user specific rules and/or steps to standardize software products or features. Think of business automation like this:  Writing software to repeat or speed a series of tasks that would otherwise require user interaction.

    Now that you understand business automation, how do you determine what steps could benefit from it?  This is actually much easier then most users think.  To get started, the next time you are working on your computer take a minute to think about the number of steps it takes to perform a given set of tasks.  (Keep in mind that almost all software support automation so the possibilities are limitless.)

    • Do you repeat the same tasks?
    • Do you make the same mistakes because the process is to complicated?
    • Is the process slow and time consuming?
    • Do you have to copy or retype data between two or more systems?

    Now that you’ve found possible cases for business automation, what is the next step?  We recommend the follow steps:

    • Go over the process from start to finish and take a few extra minutes, paying attention to the exact process.
    • Repeat the steps again, but this time make a few notes.  For example:
    1. I open Word and start a new template
    2. I copy custom information from our contact database into the Word document
    3. A picture of the requested product is added to the document
    4. We add details about the product
    5. An estimated price and delivery date is added
    6. The document is saved and exported as a PDF
    7. The PDF is emailed to the customer

    While the notes are short and may require additional details, this is the exact steps the automation will follow.  In effect, you’ve just created a project plan that a developer can work from.  Now that you have a basic plan, what is the next step?

    • The next step is working with a developer to further refine the process.  Most likely, a face-to-face or Web Conference meeting is required.
    • Following the meeting, you should have enough information to define a complete project scope.
    • The final step is to complete the business automation.

    Why website templates are a good fit for any site.

    When I metion, “Webiste Templates,” I often hear…,” We want something more unique, less formatted or cookie cutter.”  A template doesn’t mean you have to look just like everyone else.  That’s necessary when a company is trying to make it’s mark on the world wide web.  But for most, templates work!  They are the perfect starting point.  It’s common ground that you and your web designer can start from.  Once you start to select colors, plug-in content, logo’s and even modify the layout, you get a unique look to your website!  Reap the countless benefits, (and spare yourself lost time, frustration and cost) by using a template.

    If you’re not familiar with website templates, or need help finding the right design see our Website Template Gallery.

    Get a 30% discount on every screen scrape

    You’ve been asking and we are listing.  Typically we hear “I don’t need the data today but it would be nice to have”.  So we’ve decided to create a new discount program and pass the savings on to you. 

    The program works like this:

    1. You have a screen-scrape project but it’s not urgent.
    2. We review and quote the request just like any other.
    3. We put the project on a list which helps better scheduling projects.
    4. You get the same great data at a discount!

    To get started email your project details to:  sales@exdap.com

    Free web design!

    Purchase a new website plan and we’ll design the site for free. Get additional details at: Website Kick-Start details… This is a great option for small companies, new businesses or anyone on a tight budget and looking for an easy way to get on the web.

    Screen Scraper Sale

    We’re having a sale! To celebrate the release of version 5.0 of screen-scraper,  you can now get $800 off screen-scraper Enterprise Edition and $150 off screen-scraper Professional Edition. These prices will be available until August 15, 2010, after which the price for Enterprise Edition will go up to $2,799 and Professional Edition will cost $549.