screen-scraper Changelog

What's new in screen-scraper 5.0

Jul 16, 2010

feature: added REST interface
feature: can now filter out less useful proxy transactions
feature: added DataManager to facilitate saving data to a database
feature: generate multiple scrapeable files from proxy session
feature: made button bar persistent for extractor patterns
feature: retained number of lines to display for scraping session log between sessions
feature: updated scrapeable file icons to indicate when they are and are not invoked in sequence
feature: added a delete option for scraping sessions to web interface
feature: enhanced data set viewer with list view and colored tokens
feature: improved script error messages
feature: added a method to allow HTTP parameters to be removed from scrapeable files
feature: added logging levels to scraping session
feature: added ability to compare request in scrapeable file with transaction in proxy session
feature: enhanced breakpoint window to show more information, such as current script and number of scripts on the stack
feature: added syntax highlighting to extractor pattern pane
feature: added ability to pause/breakpoint a scraping session with a button
feature: extracted data can now be highlighted in last response tab
feature: pane now scrolls down when an extractor pattern is added
feature: character set can now be determined on a scraping session and scrapeable file level
feature: added ability to limit length of response for a scrapeable file
feature: enhanced handling of database backups over time
feature: can now add more session variables to a scheduled scraping session in the web interface
feature: added ability to clear completed scraping sessions from web interface
feature: enhanced a few default regular expressions
feature: properties file can now be reloaded from the web interface
feature: can now copy and paste sub-extractor patterns
feature: added ability to trim white space from extracted data
feature: added a couple of new options to invoking scripts from an extractor pattern
feature: added sutil to handle more general methods
feature: provided a way to null out session variables for tokens that didn't match
feature: provided a way to save data sets without appending to an existing data set
feature: added session.setMaxConcurrentFileDownloads
feature: added ability to install multiple screen-scraper services in Windows
feature: now higlighting selected words in script text pane
feature: added code completion and macros to script pane
feature: now using syntax highlighting in last response tab
feature: added alternative HTML tidier
feature: added notes column to proxy
feature: added getv and setv to session object
feature: now limiting script stack size in order to avoid memory problems
feature: added ability to force files to be regarded as non-binary
feature: added scrapeableFile.connectionTimedOut
feature: added find feature in proxy session
bugfix: rearranged and made redundant some GUI elements to make working with scrapeable files easier
bugfix: extractor pattern token window no longer scrolls to the bottom when a new token is added
bugfix: scrollable panes no longer scroll to the bottom when first viewed
bugfix: now retaining scroll position in panes when user selects various tabs
bugfix: the find dialog box now appears within screen-scraper's frame by default
bugfix: improved default open/save dialog box on Windows and Mac OS X
bugfix: added message if DATARECORD is absent when a sub-extractor pattern is added
bugfix: fixed resizing of child elements in breakpoint window
bugfix: fixed a bug where scraping session notes couldn't be deleted
bugfix: now clearing main panel when a folder gets deleted
bugfix: fixed a bug where a copied extractor pattern would retain script instances
bugfix: main panel is now getting cleared when a script gets deleted
bugfix: now updating list of scraping sessions for proxy session when a scraping session is renamed
bugfix: session.getNotes() was generating an exception
bugfix: pop-up windows are now appearing closer to the mouse cursor
bugfix: fixed an issue where scrapeable files couldn't be generated from certain proxy transactions
bugfix: made various fixes to proxy so that it more accurately identifies binary and non-binary responses
bugfix: fixed a bug where extractor patterns weren't being generated from selected HTML
bugfix: now clearing lower pane when proxy transactions are deleted
bugfix: fixed dataSet.writeToFile so that column headers are updated correctly
bugfix: now remembering wrap text state in scripts
bugfix: now properly resequencing scrapeable files upon deletion
bugfix: now accurately indicating when a request is multi-part
bugfix: fixed an issue where logs were being truncated
bugfix: improved handling of international characters in RemoteScrapingSession
bugfix: fixed an issue on import when character set wasn't indicated
bugfix: improved handling of hard returns in extractor patterns containing embedded variables
bugfix: improved error message on export
bugfix: improved handling of null values with data records
bugfix: will now recreate log file and continue logging when log file gets deleted
bugfix: fixed an issue where extractor patterns weren't getting highlighted properly after edit
bugfix: improved handling of large proxy transactions
bugfix: fixed an issue when resolving certain URL's from relative to absolute
bugfix: now exporting scripts that are invoked via session.executeScript
bugfix: improved handling of breakpoints in server mode
bugfix: fixed an issue where script pane wasn't being updated on import
bugfix: fixed an issue where tokens with duplicate names in sub-extractor patterns weren't being saved properly
bugfix: made running time human-readable in web interface
bugfix: user's IP address is now displayed when access is denied
bugfix: fixed a bug where the extracted data window couldn't be displayed while the breakpoint window was visible
bugfix: no longer overwriting .vmoptions files
bugfix: now using scraping session character set when exporting
bugfix: the ? character is now disallowed in object names
bugfix: logging level was always defaulting to debug when invoking scraping sessions from the command line
bugfix: can now copy text from the last request tab
bugfix: now displaying an error when invalid regular expression is entered in token
bugfix: now coloring text in log when tidying fails
bugfix: enhanced resizing of table columns
bugfix: fixed an issue where GUI would freeze up when applying an extractor pattern while scraping session was running
bugfix: made a few minor fixes to the .NET driver
bugfix: enhanced Ruby driver to be more Ruby-like
bugfix: updated XML libraries
bugfix: fixed a bug where an exception was being through when values were blank in file used by session.loadVariables
bugfix: fixed BASE HREF issue when viewing HTML in local web browser
bugfix: now deprecating unstable Windows features, including using IE as the HTTP client and allowing VBScript as a scripting language
bugfix: now allowing parentheses to be used in regular expressions, as well as back references
bugfix: fixed an issue where a script wasn't being deleted when it's parent folder was deleted
bugfix: fixed an issue where headers were being munged in certain redirect responses
bugfix: made visual sequencing of invoked scripts more logical
bugfix: improved progress bar when downloading an update
bugfix: improved placement of pop-up windows in web interface
bugfix: session.loadVariables now allows spaces before and after = symbol
bugfix: text not wrapping by default if checkbox was checked in script pane
bugfix: auto-refresh not occurring in web interface if checkbox is initially checked
bugfix: no longer requiring web.htm in web interface URL
bugfix: now highlighting button corresponding to current section in settings dialog box
bugfix: enhanced icons in Windows
bugfix: added icons for all menu items and buttons
bugfix: updated PHP class for better backward compatibility