Muse Proxy Changelog

What's new in Muse Proxy 4.4 Build 02

Jun 9, 2017
  • NEW FEATURES:
  • Muse Proxy can now create access log files in the same configurable format as those created by standard web servers such as Apache HTTP Server, format which can be set via a % style pattern (an extension of the Common Logging format). In order to do this, the LOG_FORMAT element should have the type="apache" attribute set. To have a good base for statistical information, especially in a multi-tenant environment, we recommend using more items besides Common Logging, by adding the inbound server IP address, Muse Proxy application, user session, content type:%h %A %w %W %u %S %t "%r" "%{Content-Type}o"
  • %s %b. On a fresh installation this format is already set up, while on an upgrade the old format is left in place to keep compatibility in case there are external log parsers set. If a different output is needed then more information can be found in the document ${MUSE_HOME}/proxy/doc/Muse Proxy.pdf in section "7.2 The access Log".
  • Introduced support in ProxyLoginModuleSQL.xml for SQL statements. More details on how to use this feature are present in Muse Proxy Advanced Configuration.pdf, section 6.4.5.6 ProxyLoginModuleSQL. The backward compatibility for specifying a table is kept.
  • Added support for IP ranges in ALLOW and DENY rules from ProxyLoginModuleIP.xml, and this range will be matched against the IP address the connection is coming from. Both IPv4 and IPv6 are supported. All types of rules can be mixed if need be, for example one allow/deny rule can be a wildcard such as 217.156.14.*, another rule can be a CIDR rule such as 217.156.0.0/16 and another one can be expressed using the range 217.156.11.0-217.156.15.255.
  • Introduced redirection to remote Sources depending on the end-user IP (non-proxied links). This is done via Sources.xml file via the new REDIRECT section containing IP_RULES elements which are applied on a set of sources and, if the request is for a source that matches the APPLY pattern and the request's end-user IP satisfies the ALLOW/DENY sequences, then the response will be a native redirect to the source URL.
  • Source parameters can be provided via Sources.xml level in the element via multiple children. Each parameter can be further referred in the Source Profile in or via ${name} syntax and its value will be resolved to the content of this node, exactly as if it were defined inside the source profile.
  • The Single Sign-on Authentication (other than SAML) core was upgraded and part of the new features Central Authentication Service (CAS) is also supported. Upgrades instructions related to how to handle the changes in securityContext.xml are provided via the setup.
  • Keep up with SHA1 deprecation.
  • To avoid DNS limitation of 63 bytes per label in case of proxing https:// hosts via Rewrite by Host technique using https:// proxy URLs, there is now the possibility to map the very long FQDN to shorter names by using HOST_MAPPING elements in the source profile.
  • This release contains an improvement for Multi tenant environments using the same host name for all tenants but individual IPs for each one. A chaining request to proxy itself on the different IP is no longer taking place for each request, rather the outbound IP is set directly to the one of the chainingproxy.Toachievethisonlythe PROXY_HOSTmustbeconfiguredtotheallocatedIPin the source profile or at the application level and the PROXY_PORT must remain empty.
  • The generation of the Client Session cookie values was improved. The generation of the Connection ID values was also improved.
  • Implement an improvement when action=source is called, and data is read from the Sources.xml application file.
  • Refine the errors "Unexpected exception while accessing target source." to contain more details about what was wrong with the access.
  • Added a limit for the size of multipart POST requests which will be kept in memory and requests which will exceed this limit will be temporary saved on the file system. The value of the limit is configured in USE_MULTIPART_TMP_AFTER tag from ${MUSE_HOME}/proxy/MuseProxy.xml.
  • Added COOKIE_PASS_PATTERNS options and cookies with name matching these patterns are passed into the browser even if they have a domain. This configuration must be used with care and only where strictly required. Cookie name patterns can be specified such as SESSION*, separated by semicolon (;).
  • Secured the /admin context availability.
  • Related to proxy application IP Authentication, the REVERSE flag was added to control if reverse DNS is performed for the end-user IP. The trend is to set it false in the configuration files from now on. Reverse DNS is too costly and can slow down the authentication process.
  • In case of HMAC, Referer and IP login modules there's no ID to check against a database and hence nothing to be written in the log file, however in case of various integrations we could be receiving a special parameter in the request for tracking purposes and we want to keep this for logging.
  • Because JSESSIONID name is too general, the session cookie names for the embedded Jetty
  • contexts (related to Single sign-on) were changed.
  • Added encodeURIComponent and decodeURIComponent to be used for reference and parameter process for first source requests (including extract and navigate scenarios). The functions are compatible with the JavaScript ones. The existent encodeURL and decodeURL are based on JDK URLEncoder/URLDecoder which are using application/x-www-form-urlencoded MIME format which is not entirely the same as the URI encoding which, for example transforms space into instead of +, for example and some servers are sensitive to these differences.
  • Bug fixes:
  • Microsoft Azure AD OpenID Connect End Point v2.0 and End Point v1.0 can now be used for authentication without complex workarounds - guidelines and suggestions are available in the Ad ministrator Console in Configuration / SSO Authentication.
  • Security constraints for SAML or SSO authentication when starting from plain http:// proxy links are by default not enforced.
  • HttpModuleApache is now correctly sending post data for the extract and navigate scenarios as URL encoded data.
  • Filtering on the exact Client Session ID in the Administrator Console was corrected.
  • Fixed parameter name encoding for HMAC Link Generator page inside Administrator Console.
  • Avoid redirect loops when a Type 2 (rewrite by path) link expires for SSO2 (OAuth / OpenID) authentication.
  • Correctly persisting Tiny URLs, that are used in some cases for MuseKnowledge Search integration.
  • Fixed memory usage when downloading more log files from the Administrator Console.
  • Quicker release of file descriptors when local resources such as images, javascript or css are served by Muse Proxy.
  • While rewriting object, embed and param elements the protocol relative URLs (the ones starting with //) are correctly treated.
  • The starting point Type1 links used for MuseKnowledge Search integration can also be generated on https:// protocol.
  • Existing navigation session were ignoring the FIND/REPLACE filters after a mnm.jar update - this is now fixed.
  • A very rare deadlock appearing when a Muse Proxy under extremely heavy usage has its mnm.jar updated. This fix is actually carried by mnm.jar version 1.513 itself, so old versions of Muse Proxy can be updated without a full upgrade to fix this.
  • Interpreting the request Forwarded headers was failing if no proto= was found in any of these headers. This is now fixed.
  • The following rare case was fixed: if the same linkout source is in two different applications and the same end-user accesses both applications from the same browser at the same time it is possible that the source we link to is used via a navigation session from the other application yielding misleading statistics.
  • Protocol relative URLs in HTTP redirects could have generated wrongly rewritten links - this was fixed.
  • In case no ENCODING is defined in the source profile, first source request(s) (such as extract and navigate) are now considering UTF-8 as a default encoding for various processing such as parameter encoding processing or deserialization from gzipped content (also when no charset is present in the Content-Type reply).
  • All multiple HTTP response header fields are now reaching the browser in the response to the first rewritten URL (by host or by path) request which is initiated by the source request (?action=source, ?url=).

New in Muse Proxy 4.3 Build 02 (Dec 9, 2016)

  • New features:
  • In the family of Single sign-on authentication, besides SAML2.0, MuseKnowledgeTM Proxy now supports a wide range of OAuth, OAuth2, OpenID Connect SSO based. MuseKnowledgeTM Proxy supports connectivity with more than a dozen of OAuth providers and also a generic OAuth client implementation can be configured for authentication to the providers that are not diverging from the usual practices in OAuth requests and responses (e.g. return the access token in JSON as "access_token" : "{value}", return profile in JSON and not XML, use "code" and "state" parameter names, no additional hashes with the access token). The existent OAuth specific support is for: BitBucket, DropBox, Facebook, Foursquare, Github, Google, LinkedIn, Odnoklassniki, ORCiD, Paypal, Strava, Twitter, Vk, Windows Live, Word Press, Yahoo. Note that Google ensures authentication with both the public gmail.com domain as well as Google hosted institutions via Google Apps for Education, for example. There's also a general con figuration for any CAS server using OAuth protocol and a general support for the providers that are following the usual practices as described previously. For helping with OAuth configuration, a new section was added into the MuseKnowledgeTM Proxy Administrator Console, Configuration menu - SSO Authentication. It contains a checklist and detailed guidelines to configure OAuh for a MuseKnowledgeTM Proxy Application.
  • External HTTP Authentication Login Module for MuseKnowledgeTM Proxy is available. There may be cases in which we need to authenticate against an existent HTTP service or even a html login form from the intranet. MuseKnowledgeTM Proxy front end logon is presented but behind the request to a remote HTTP login end-point is made and a success/fail decision based on elements from the page is taken. There is no special need of introducing extra text in comments as long as there are clear elements that confirms a success/failure. Also the sources group ID can be extractable or inferred based on group names, messages, elements.
  • Introduced experimental support for load balancers that do not spoof/masquerade the IP of the end-userandpassitintheprotocollayerviaHAProxyPROXYProtocolv1orvia X- Forawarded-For. The IP of the end-user is needed for authentication and logging purposes.
  • Changes:
  • Use the ALLOW_PROXY_PROTOCOL options to specify the IPs or address templates separated by semicolon (';') that are allowed to send HAProxy PROXY protocol v1 information. If MuseKnowledgeTM Proxy receives HAProxy PROXY protocol v1 it first checks the source address to see if it trusts it. Note that SSL must be terminated at the load balancer side in case of usingHAProxyPROXYProtocolv1.Asimilaroption, ALLOW_X_FORWARDED_FOR,isavailable in case of using X-Forwarded-For. Although the protocol layer end-user IP is used for all au thentication and access logging purposes, the IP of the load balancer can still be observed under the connection entry log 110 from MuseProxyStatistics.log files, because that entry is logged when a socket is opened and not when the protocol is understood. However the statistics entry 210 (used when new data is received on a connection) was extended to include the end-user IP from the protocol layer.
  • Category grouping for source layer presentation is now possible. Multiple areas can be defined, including A-Z ones and these are displayed in different tabs. Integration with MuseSearch passthrough is available if dblist source attributes are defined.
  • Introduced experimental support for follow-up links without authentication (session cookie) such as for the preflight OPTIONS where the standard requires the browser to avoid sending au thorization data. Only links that are generated by a valid navigation session are allowed and only if the HTTP method and link matches the new source configuration element value.
  • Apache HTTP Client library can now be used for the first source request (extract and navigate scenario). Because the Oracle JDK URLConnection does not allow the control of the outbound IP address up to now we were forced to perform an extra request through MuseKnowledgeTM Proxy and this increase the complexity of troubleshouting and authentication configuration and adds an extra request. The Apache HTTP Client allows control over the outbound IP address and there's no need of an extra request. To configure a source to use the Apache HTTP Client edit Sources.xml from proxy application level and instead com.edulib.muse.proxy.application.sources.modules.impl.HttpModu le use com.edulib.muse.proxy.application.sources.modules.impl.HttpModu leApache.
  • Added a limit which usually triggers using temporary files for saving streams of bytes in certain cases, for example for performing gzip. Otherwise these operations are performed in-memory, and, although more time-efficient, this can limit the number of concurrent requests. This controlled via the new flag USE_TMP_FILE_STREAM_AFTER in ${MUSE_HOME}/proxy/webcontexts/NavigationManager/profiles/NavigationSe ssion.xml.
  • Added a new tool in the MuseKnowledgeTM Proxy Administrator Console - HMAC Link Generator - for generating HMAC links for testing the login via HMAC (keyed-hash message authentication code) signing. The utility allows specifying all possible parameters and combinations for generating a HMAC link.
  • Added a new tool in the MuseKnowledgeTM Proxy Administrator Console - Evaluate Regex - for evaluating regular expressions. The utility is most usefull for administrators to troubleshoot sources filter configurations for find and replace. It has two forms: By JDK RegEx and By Running Filter. The By Running Filter tool generates the XML snippet that can inserted into the MuseKnowledgeTM Proxy source profile.
  • For load balanced environments the ID value (if defined in MuseProxy.xml and if cookieSuffix="true" - which is true by default, if missing) is also added to the session cookie name as a suffix (e.g. MuseProxySessionIDp2), because this behaves more reliable in certain Load Balancer cases, such that the ones combining routing rules (/admin rule or .p1.[a-z].http rule to go to a certain proxy) with the sticky cookies mechanism.
  • Log files can be named containing elements of the creation date, based on the new pattern attribute of the LOG element. For example to capture the activity on a daily basis in files such as access- 20161123.log and keeping a maximum of 365 such files the corresponding logger will have to be configured as below in MuseProxy.xml.
  • Bug fixes:
  • The PUT HTTP method was relayed as POST and some AJAX implementations may not accept this. This was corrected.
  • A fix related to file uploads through rewritten POSTs was considered.
  • Digest authentication for remote sources was not working correctly if qop (quality of protection) value is not wrapped in quotes. Although this behaviour was according to the RFC, MuseKnowledgeTM Proxy is now more permissive in this regard.
  • IP rules for the application level ProxyLoginModuleIP login module are treated the same way the rules for ${MUSE_HOME}/proxy/hosts.xml are, that is the first rule that match counts.

New in Muse Proxy 4.0 Build 02 (Jan 8, 2015)

  • Bug fixes:
  • Content-type not seen as "gzipable" by Muse Proxy during the content processing operation could
  • end up gzipped twice. Such a case was discovered for the text/json-comment-filtered
  • Content type. This is now fixed and if Content-Encoding
  • is still present in the reply after
  • Navigation Filter did the processing then we don't perform the gzip action