What's new in Apache Airavata 0.14
Jan 9, 2015
- Tasks:
- HostAddress in ComputationalResourceScheduling is required.
- Remove airavata-client-api dependency from other module
- Zookeeper Implementation to Advertise Available/up and running Airavata Services/Components
- Modify Airavata Distribution to Enable Separate Distributions for Orchestrator, Gfac and Xbaya.
- Replace legacy application descriptors used in gfac with app-catalog
- Sub-tasks:
- Support option with parameters for GramProvider
- Craft the Workflow Interpreter CPI
- Updated Xbaya to enable execution of a single application through thrift api
- Submit real time job count details to zookeeper
- Add a inmemory job count map to Orchestrator
- Add throtle-resources flag to airavata API
- Implement RabbitMQ publisher
- Provide a way to enable and disable Job count validation
- Cleanup Airavata property file for old service properties
- Remove connection to Workflow service from Xbaya
- Story:
- Update Airavata JGLOBUS and Unicore jar
- New Feature:
- Add ResourceJobManger add, update, get, delete methods to airavata-api
- Improvement:
- Support For-each for Local providers
- Implement logger in WorkflowInterpreter
- API need enhancement so user can set a user identifier into Context header
- Multiple Security context to run a grid job
- ActualParameter and DataObjectType have same objective
- Can we have a way of searching for all the projects, experiments in the gateway?
- Airavata API function to get Job details
- API function to get data transfer details.
- Input data transfer details are not updated
- Allow multiple client to move the data using same server
- Get partial results incase outputs are not produced.
- RabbitMQ messaging is not supported for SGE resources
- Update log messages to track life cycle of experiments , jobs and tasks.
- XBaya doesn't support multi-app workflows with new Thrift API
- Remove Registry.wsdl stub autogeneration
- Change return type of jobSubmission API methods to string
- Add API methods to get job submission information
- Bug:
- Error message not sent to Xbaya as fault
- Provide GridFTP URL incase of URL and URLArray type
- Unable to set a different gateway name other than "default"
- No free ports available to run GRAM provider and GridFTP transfers
- PersistenceException while executing BES or Gram Provider
- Update Client samples READMEs and INSTALL files
- Command for SLURM job cancel need correction
- Error details are saved twice which adding two records in the database
- Output path is not getting set to output context incase we scan output directory
- App catalog database connection timeout issue
- Orchestrator context does't set initial GFac instant list correctly in one of it's constructor.
- GSISSHAbstractCluster not throwing error if there is issue submitting job
- NullPointerException to create a SSH connection
- Rewrite CreateLaunchExperiment class in order to work with registerApplication method
- Experiment Status is updated to completed on job complete
- Host Aliases and IpAddresses not getting saved in Compute Resource Object
- AdvancedSCPInputHandler is not working if we need to transfer data to BigRed
- Experiment Status is not updated to Complete
- SCP to a newer version of Centos VM does not work
- SSHInputHandler cluster object is null
- Error message when configuring XBaya input parameter configuration
New in Apache Airavata 0.13 (Aug 11, 2014)
- Whats new in this release:
- A key addition in this release is addition of App catalog. The app catalog incorporates registering applications, computational resources and flexible use of job management and data movement protocols.
- More Details:
- [AIRAVATA-1125] - Application Catalog
- [AIRAVATA-1067] - Allow file transfer from the gateway to the resource through Airavata
- [AIRAVATA-1015] - Craft data models for application catalog
- [AIRAVATA-1236] - Database design for Application Catalog
- [AIRAVATA-1302] - Implement App catalog open-jpa layer and resource layer
- [AIRAVATA-1303] - Start app-catalog database with app catalog thrift service starts
- [AIRAVATA-1305] - Adding test cases to test app-catalog cpi
- API Changes:
- AppCatalog API was added as described above.
- No changes were made to previously release API methods.
- Improvement made in this release:
- This release bundles a stable iteration of using Apache thrift as a framework to implement Airavata API.
- More Details:
- [AIRAVATA-481] - GFac Sample
- [AIRAVATA-958] - Integrate airavata credential store with GSISSH library in order to retrieve credentials
- [AIRAVATA-989] - Clone a created experiment in Orchestrator to launch on different host
- [AIRAVATA-1070] - Integrate the SSH file transfer between the gateway and resources to the Gfac
- [AIRAVATA-1126] - Application Catalog Design
- [AIRAVATA-1235] - No error type defined for a missing Project
- Issues fixed in this release:
- Testing and bug fixing resulted in the following fixes.
- [AIRAVATA-437] - Stdout and stderr files are read as string and will be problem if the file are big
- [AIRAVATA-743] - Cleanup database sql scripts
- [AIRAVATA-950] - Host parameters not saved in the database
- [AIRAVATA-951] - Misleading error message in airavata server console when submitting a job to BR2
- [AIRAVATA-975] - Wrong exception given when certificate path is configured incorrectly
- [AIRAVATA-1026] - GFacAPI logic to load input/output handlers is not working property
- [AIRAVATA-1058] - Airavata 0.11 Source Build: Dependency Problems
- [AIRAVATA-1085] - Update the Experiment Status
- [AIRAVATA-1119] - JPARegistry tests are failing
- [AIRAVATA-1310] - When entered gateway username and password user gets redirected to XSEDE user portal
- [AIRAVATA-1338] - Cannot create Experiments using PHP Gateway
- [AIRAVATA-1345] - Refresh button for Experiment Status page
- [AIRAVATA-1347] - Cannot edit or view summery of experiments created with WRF application and its input files attached
- [AIRAVATA-1348] - Cannot Save and Launch WRF experiment at the same time. It fails.
- Client Packages:
- A key client side focus of this releases is to validate Airavata functionality with PHP clients. a PHP Reference Gateway is developed to exercise API functionality.
- PHP command line samples also have been developed and tested against the API.
- An initial version of the C++ SDK was also contributed but is not yet fully tested.
- More Details:
- [AIRAVATA-1064] - Create php Samples using Airavata Thrift API Client SDKs
- [AIRAVATA-1352] - In PGA gateway, include creation time in Experiment Summary
- Upgrade Instructions:
- Airavata 0.13 should work with the previous 0.12 release.
- Older releases will need to be manually updated. Beyond 1.0, Airavata will maintain backward compatibility within major releases.
- Known Issues:
- Airavata 0.13 continous to be a stepping stone towards a stable 1.0 release. A known limitation is lack of a UI to register applications (using the app-catalog API).
- XBaya and the workflow capabilities are being integrated into the new architecture and do not yet work with this release. A subsequent release (0.14) will bring back workflows into a functioning.
- Documentation:
- This release was used as a tutorial to outreach apache airavata to science gateway developers.
- Tutorial documentation - https://cwiki.apache.org/confluence/display/AIRAVATA/XSEDE+2014+Tutorial
- [AIRAVATA-858] - Error data retrieval from the Airavata API
- [AIRAVATA-1329] - Document Hands on tutorial material on PHP reference gateway
- [AIRAVATA-583] - Improve content coordination across website and wiki
New in Apache Airavata 0.11 (Jan 7, 2014)
- Improvement:
- [AIRAVATA-278] - Improve GFac stdout handling to cover output types
- [AIRAVATA-941] - JCR properties in GFAC's service.properties are obsolete
- [AIRAVATA-952] - Add credential store to the Airavata API
- [AIRAVATA-963] - Refactoring Integration Tests module to allow easy extendability and remove redundancy
- Task:
- [AIRAVATA-953] - Distribute JNLP With Releases
- Sub-task:
- [AIRAVATA-925] - Display status of experiments upfront in experiments list
- [AIRAVATA-938] - Update XBaya window to support file transfer
- [AIRAVATA-967] - Update Unicore client version
- Story:
- [AIRAVATA-965] - Update Airavata JGLOBUS and Unicore jar
- Bug:
- [AIRAVATA-922] - Reduce the size of war distributions
- [AIRAVATA-954] - Evaluating thrift based execution API
- [AIRAVATA-969] - GSISSH provider not fail on PBS Syntax errors
New in Apache Airavata 0.10 (Nov 9, 2013)
- Sub-task:
- [AIRAVATA-760] - GSISSH provider to run simple job manager commands provided by the user
- [AIRAVATA-761] - Input output data transfer using gsiscp or GridFTP
- [AIRAVATA-763] - Extending GFAC schema to support GSI hosts configurations
- [AIRAVATA-928] - Prepare release
- Bug:
- [AIRAVATA-744] - Cleanup configurations files
- [AIRAVATA-756] - Ensure Airavata can renew proxy for long running jobs.
- [AIRAVATA-882] - Tomcat port configuration accessing problem in certain older tomcat 7 version
- [AIRAVATA-920] - XBaya initialization fails when it is initialized from another client application
- [AIRAVATA-931] - GSISSHProviderTest is not working
- [AIRAVATA-933] - Regex in OutputUtils does not support space in output string
- [AIRAVATA-939] - When workflow fail it goes to workflow completed mode and not showing the red node
- Improvement:
- [AIRAVATA-870] - Enable CORS support in Airavata server
- [AIRAVATA-880] - Have a UserRegistry API to support user related operations
- [AIRAVATA-912] - [GSoC] JSON equivalent of XWF files
New in Apache Airavata 0.9 (Sep 24, 2013)
- New Feature:
- Integrate Credential Management service to Airavata Server
- Task:
- Negotiate with Apache Infra and increase the distribution size to 200 MB
- Prepare release
- Improvement:
- Remove invoker caching from Workflow interpreter
- Airavata API to get JOBID of GramJob based on workflow service node name
- Upgrade tomcat version used in integration-tests
- Upgrade JGlobus version to 2.0.6-RC2
- Handling of URI and URIArray containing different files with the same name should be handled correctly
- Bug:
- Confirming the airavata shutdown is completely graceful
- Airavata Client Exception handling
- Error message when I try to delete Application Service from Xbaya dashboard
- When registering urls in registry, we need to register based on transport protocol
- Xbaya does not give error messages when connecting to database with wrong credentials
- Fixing the provided samples
- Levenshtein distance sample is not working
- Fix the context root specified in the airavata-server.properties file for Airavata webapp
- Exception in jpa-registry when building in ubuntu with java 64 bit
- SSHProvider not working in trunk
- Scheduling info set from advanced options in client-side is not recovered properly at server-side
- Erroneous output is set as the input to the next node from the workflow interpreter
- EmbeddedInvoker is not parsing previous node output to the next node input correctly
- Experiment Data list return from getAllExperimentMetaInformation() method of provenance manager not sorted according to experiment submitted date
New in Apache Airavata 0.8 (Jul 24, 2013)
- New Feature:
- [AIRAVATA-862] - Introducing a HostScheduler interface and allow users to write their own HostSchedulers and register them with Airavata
- Task:
- [AIRAVATA-93] - Review gfac-core code
- Improvement:
- [AIRAVATA-450] - Delay in JobSubmissionListener after getting status from Gram
- [AIRAVATA-827] - Extending Gram RSL parameters to provide Job Name, email options
- [AIRAVATA-832] - Provide support to configure a new provider in gfac-config.xml
- [AIRAVATA-850] - Add a UI component to EC2 menu which allows a user to generate a key for EC2 communication
- Bug:
- [AIRAVATA-304] - WorkflowInterpreter had some memory leak which can be seen when send requests continuously upto 4000
- [AIRAVATA-721] - Xbaya attempt connection at a down Airavata Server
- [AIRAVATA-745] - Levenshtein Distance Sample Needs to be updated
- [AIRAVATA-746] - Airavata server populates incorrect URLs (GFac, interpretter ...) when HTTPS is enabled in tomcat
- [AIRAVATA-818] - Set host dynamically per service request
- [AIRAVATA-824] - EmbeddedGFacInvoker is not reading hostname from application scheduling context
- [AIRAVATA-826] - Tar data uploading using gridftp cient in GFAC is not complete
- [AIRAVATA-829] - In Airavata API, HPCSettings the maxWallTime is not working
- [AIRAVATA-833] - GridFTPOutputHandler is not able to download outputs from outputData folder incase of URI data type.
- [AIRAVATA-834] - Add support for GridFTP output file staging
- [AIRAVATA-836] - In GramRSLGenerator setOutputDataDirectory to context output path is not correct
- [AIRAVATA-837] - Not able to load property incase of provider properties defined in gfac-config.xml
- [AIRAVATA-849] - Error messages from Airavata API
- [AIRAVATA-851] - Use the username specifed in the workflow graph when sshing to the AMI instances
- [AIRAVATA-856] - Make credential-store specific table scripts lower case
- [AIRAVATA-857] - Error while trying to query the experiment data after few hours
- [AIRAVATA-861] - Integration tests are failing
- [AIRAVATA-865] - Renaming GFac Job related functions in the rest service and the rest client
- [AIRAVATA-869] - When the application aborts with return value of minus Status of failure is not reported in to registry
- [AIRAVATA-872] - ws-messenger services come to inconsistent state with a failure scenario
New in Apache Airavata 0.7 (Apr 16, 2013)
- Story:
- [AIRAVATA-530] - Implement a new Handler architecture
- New Feature:
- [AIRAVATA-315] - Provide an interface for Gateway administrator
- [AIRAVATA-316] - Provide a way to access the log for each experiment
- Improvement:
- [AIRAVATA-202] - Add EC2 Provider to Airavata
- [AIRAVATA-540] - There's no way to find final workflow outputs from notifications
- [AIRAVATA-780] - Add support to add multiple security context
- [AIRAVATA-782] - Add the EC2 Job Submission Schedular to the trunk
- [AIRAVATA-787] - Add support for Unicore and SSH provider in Xbaya UI
- Task:
- [AIRAVATA-471] - Add sample workflows to demonstrate the use of foreach construct
- [AIRAVATA-757] - Remove dependency to OA4MP snapshot jars once OA4MP 1.0.6 is released
- [AIRAVATA-803] - Run Amazon EC2 workflows through XBaya
- [AIRAVATA-804] - Fix the mechanism to keep the AWS secret key and access key within the workflow.
- Sub-task:
- [AIRAVATA-531] - Writing new Context classes to keep all the Context data
- [AIRAVATA-770] - Add Unicore provider to submit jobs
- [AIRAVATA-783] - Brief diagram about authentication framework
- Bug:
- [AIRAVATA-635] - Providing sample to configure workflow context header for each node
- [AIRAVATA-742] - Make authenticators.xml read default database configurations from airavata-server.properties
- [AIRAVATA-747] - SSH provider in EmbeddedGFacInvoker
- [AIRAVATA-748] - Dependencies missing for SSH provider
- [AIRAVATA-749] - Porting GSOC changes to Airavata trunk
- [AIRAVATA-750] - airavataAPI.getProvenanceManager().getExperimentDataList()
- [AIRAVATA-751] - Nullpointer exception in SSHProvider.getSession()
- [AIRAVATA-764] - Job should not be canceled if there is a job error.
- [AIRAVATA-765] - GridFTPOutputHandler need to return file url incase output type in URI
- [AIRAVATA-772] - Cannot start Airavata on Windows
- [AIRAVATA-773] - XBaya does not work on Windows
- [AIRAVATA-775] - Add the EC2 UI components to XBaya
- [AIRAVATA-778] - Xbaya workflow editor: "Airavata Registry" panel doesn't update after making changes
- [AIRAVATA-781] - Add a test case to gfac core to submit EC2 jobs
- [AIRAVATA-799] - Gram job data storing failing
- [AIRAVATA-801] - 10 mins sample failed in 0.7 RC2
- [AIRAVATA-809] - EC2 Instance Management window is not properly functioning due to backend issues
- [AIRAVATA-812] - Add documentation for EC2 job submissions through XBaya
- [AIRAVATA-813] - Incorrect instructions given in the wiki for Sample Workflow SimpleForEach
- [AIRAVATA-815] - Issues when terminating and starting instances/AMIs using Amazon instance management window
- [AIRAVATA-816] - Workflow Run sample fails because of wrong host
New in Apache Airavata 0.6 (Jan 31, 2013)
- New Feature:
- [AIRAVATA-387] - Support for individual users in a community
- [AIRAVATA-572] - Add basic auth based authenticator to Airavata
- [AIRAVATA-573] - Add support to connect different types of user stores to authenticate Airavata REST API
- [AIRAVATA-631] - Change Airavata API (airavata-client) to use Airavata REST Service
- [AIRAVATA-632] - Change XBaya to use the Airavata Client
- [AIRAVATA-633] - Change XBaya to use Airavata Client for Workflow Launches
- Improvement:
- [AIRAVATA-539] - Provide WorklfowNodeID in wokflowInvoked Notification
- [AIRAVATA-644] - Change default authenticating database to Derby
- [AIRAVATA-645] - Provide a utility method to set basic access headers in client side
- [AIRAVATA-653] - Use Embedded Tomcat as servlet container for Airavata
- [AIRAVATA-658] - Create a context class to store request specific properties
- [AIRAVATA-683] - Improving Application Manager API
- [AIRAVATA-684] - Add stop monitoring functionality for the Monitor object
- [AIRAVATA-685] - Improvements to the WorkflowManager API
- [AIRAVATA-704] - BPELScript log messages to parse wsdl should be of debug level
- [AIRAVATA-719] - Enable rest client to communicate over HTTPS
- Task:
- [AIRAVATA-556] - Adding REST interface to Airavata Registry API
- [AIRAVATA-670] - Update LICENSE & NOTICE files in distribution
- [AIRAVATA-677] - Simplify runExperiment function overloading in Execution Manager
- [AIRAVATA-679] - Make the WorkflowContextHeaderBuilder transparent to the API User
- [AIRAVATA-698] - Move the registry related property data included on airavata-client and airavata-server properties files instead
- [AIRAVATA-726] - Generate a WAR distribution for Airavata Server at build time
- Sub-task:
- [AIRAVATA-65] - Update Audit Notifier
- [AIRAVATA-66] - Update Performance Notifier
- Bug:
- [AIRAVATA-259] - A notification is sent out with the description "Invoking a workflow" for each service invocation this is misleading.
- [AIRAVATA-528] - During workflow data saving node with a name of "experiments[1]" creating
- [AIRAVATA-619] - MsgBoxWsaResponsesCorrelator is creating an infinite thread and there's no way to interrupt that thread
- [AIRAVATA-621] - WorkflowInterpretorEventListener is accumulating - not properly shutting down
- [AIRAVATA-622] - Error in console when running a workflow- Not always
- [AIRAVATA-628] - Add new user displays user add to JCR registry
- [AIRAVATA-643] - Add different registry accessors to airavata-server.properties and update AiravataJPARegistry accordingly
- [AIRAVATA-646] - Make tests work with paths that include spaces (Fix build failure in tests due to paths with spaces)
- [AIRAVATA-648] - MsgBoxWsaResponseCorrelator thread is not returning
- [AIRAVATA-650] - Provide a sample how to push notification messages to someother endpoint after capturing them in airavata client
- [AIRAVATA-652] - Create Java REST client to REST service
- [AIRAVATA-654] - Cannot start Airavata if the installed path contains spaces
- [AIRAVATA-664] - Null pointer exception is thrown from REST service when calling the registry API
- [AIRAVATA-672] - Adding embedded tomcat as default server to Airavata
- [AIRAVATA-680] - Application Service menu get repeated in XBAya GUI
- [AIRAVATA-681] - Enable hash passwords in Airavata local user store and make SHA as default password hash method
- [AIRAVATA-682] - Failed login when connecting to an existing database
- [AIRAVATA-690] - Lonestar GridFTP URL is wrong in default host objects
- [AIRAVATA-692] - Not able to set HPC Configuration on New Application Deployment screen. Urgent!!
- [AIRAVATA-693] - Edit Application- Edit deployment link does not work
- [AIRAVATA-694] - Edit Application > Edit deployment > Scratch working directory empty
- [AIRAVATA-696] - Airavata-client needs database configuration in client side code which is wrong
- [AIRAVATA-699] - When executing workflow user is authenticated multiple times for the same session
- [AIRAVATA-700] - Edit Host Description dialog vanishes once one description get deleted
- [AIRAVATA-701] - Maven clean install does not remove existing target folders for xbaya-gui
- [AIRAVATA-705] - There are some dependencies missing in Airavata client distribution
- [AIRAVATA-706] - Updating the HPC Configuration Options
- [AIRAVATA-708] - XBaya doesn't return when backend gfac operation fail
- [AIRAVATA-713] - Cleaning up unnecessary dependencies from Xbaya distribution and airavata-client distribution
- [AIRAVATA-715] - Database setting are picked up from registry.properties.
- [AIRAVATA-718] - An exception while starting integration tests
- [AIRAVATA-724] - A valid SimpleMathService is not available for the 5 minute tutorial
- [AIRAVATA-727] - Airavata distributions are not installed to the maven repository
- [AIRAVATA-728] - OutputDataSettings does not have method to set node information.
- [AIRAVATA-729] - Advanced options are not saved when creating an application
- [AIRAVATA-734] - Multiple Workflow is invoked messages in the workflow
- [AIRAVATA-741] - Airavata startup fails when Airavata is pointed to a new database