<alt> Mobile Blog

The Audible Web in Web 3.0

Sun, 6 Apr 2008 18:41:00 -0400

This entry is all about Audible Web support in our Web 3.0 products.

My last entry discussed our support for speech synthesis in our Web 2.0 products. We looked at how the Mashup server implements text-to-speech as well as how our Mashup clients access the Mashup audio.

If we look at Semantic Web concepts, we can see that it envisions users accessing a web of data were users can access just the bits and pieces of a page that the user wants-- as opposed to today's web where you get the whole page with ads and stuff that you don't care about. Read the entry entitled Semantic Web: Documents vs. Elements for more information.

As you know from the Semantic Web series, we have implemented the SemWeb vision through our Web 2.0 Mashup technologies. You also know that we had to shun most of the Semantic Web standards prescribed by the W3C. This was discussed in the post Semantic Web: W3C vs. WWW.

So where does the Audible Web fit in the Web 3.0 data web? In our meta data implementation, we provide direct support for the Audible Web.

The meta data that describes those bits and pieces of a page that are important to a user also contains an SSML version of the data. SSML is the W3C standard format to encode text-to-speech instructions. The SSML can be fed to a speech engine directly or transcoded into the proprietary format used by non-standard speech engines.

The great thing about SSML is that it supports the encoding of languages other than American English. So even though we generate American English text-to-speech markup, a user or service provider can add other languages to the meta data. This is doable since in our implementation of the Semantic Web, meta data is created and owned by users and not the publisher.

More information about our meta data implementation is found in the blog entry named Semantic Web: Triple vs. Grand Slam.

I'll conclude with an example of how the Audible Web should be intrinsic to the Web 3.0 data web-- and is in our implementation:

Wishing to spread information about the plight of his fellow countrymen, a Tibetan web user creates a Mashup from an English language news source and attaches a text-to-speech translation to the meta data. The meta data can then be processed by the MARY text-to-speech engine. Users can verify and enhance the translation because the English language HTML fragment is embedded with the text-to-speech version. Alternatively, an audio file can be attached if the text-to-speech engine does not support the language directly.

Did you hear that? It's the Audible Web arriving.

The Audible Web in Web 2.0

Sun, 6 Apr 2008 13:29:14 -0400

Commonly called text-to-speech or just TTS; speech synthesis technology is integrated into our Web 2.0 and Web 3.0 products.

Speech synthesis is a convenience feature to power users and a necessity to the visually impaired: we are pleased to support both types of Mashup users.

First debuting in May 2006 in support of Opera Widgets; we enabled widget users to hear their Mashup content. It's a very powerful capability. Your widget not only displays your specialized content but also you can hear the results even when you're not watching the screen or if your widget is buried under layers of windows.

The mechanics are conceptually straightforward: the Mashup widget runs a JavaScript timer and uses Ajax to request new content from the Mashup server. The Mashup server fetches the remote web snippet(s)-- transforming and aggregating as needed-- then generates a text version of the HTML content which is passed to a speech engine and out comes an audio version of the content which is stored in a file.

The text version of the HTML content preserves the HTML element structures including lists, tables and images, etc. (Even the leading text-to-speech services used in blogs and online newspapers do not support the richness of HTML content. They simply read the text content and bypass complex elements such as lists, images, and tables. On the other hand, we do it all.)

The widget displays the Mashup content and reads the audio file out loud. All the code for the widget including the HTML, CSS, JavaScript, Ajax, and playing the audio file is generated by the Mashup Designer for Opera Widgets.

Nowadays speech synthesis of Mashup content is supported in our Mashup clients including Safari on iPhone, Apple Dashboard Widgets, Microsoft Vista Sidebar Gadgets, IE 8 WebSlices, Mozilla Prism, Opera Widgets, Sun's Lively Kernel, and Sun's JavaFX.

You're probably thinking why we didn't include our Adobe AIR Mashup client in the list of text-to-speech enabled clients. Well... it seems that support of common internet audio formats are... unsupported. According to Adobe's site, these are the only supported audio formats for external content:

So, why don't we encode the Mashup content in mp3 files? A good question which segues into our Mashup server and its underlying speech synthesis technology. We've implemented support for 2 speech engines: Free-TTS and Apple speech.

Developed by Sun Labs, the all-Java open source Free-TTS is our default speech engine. Its voices are middle-of-the-road and that means they sound robotic. But it works on all platforms which we support: Solaris, Linux, Windows and the Mac. On the Mac, we use Apple's speech engine which probably has the most natural voices in the industry.

Both speech synthesis engines do not output to mp3 formats but do support AIF. Consequently, AIF is the default audio format for the Mashup server. So we've decided to skip supporting Mashup audio on Adobe AIR.

Now to the server. To instruct the Mashup server to output an audio version of the Mashup results, use the URL parameters use-TTS and audio-id. An example URL is:  http://10.37.131.2?mode=webSlice-ie8&use-TTS&audio-id=123 
The value of the audio-id parameter will be the name of the audio file.

In my next post, I'll discuss the Audible Web in the context of Web 3.0 so stay tuned...

Semantic Web: W3C vs. WWW

Fri, 4 Apr 2008 03:47:06 -0400

This post summarizes the main concepts from the Semantic Web series. So let's restate what the <alt> Semantic Web provides:

Meta data that's created by users and not page authors
Meta data that's used for user empowerment and not publisher control
Meta data that describes the document's content and not the document itself
Meta data that's dynamic, active, and executable and not static and passive
Meta data that empowers all types of users including the vision impaired and non-American English speakers
"Knowledge" that is based on what the user deems useful from 1 web source or the composition of many web sources and is not divined by special databases and query languages

To sum up, the <alt> Semantic Web implementation differs from the W3C prescribed implementation because at its core: the <alt> Semantic Web empowers web users and the W3C Semantic Web empowers everyone but users.

A bold statement. Heresy to some. Reality to the enlighten.

This concludes our first series on the Semantic Web. We will add new entries to update you as our implementation advances or... if the W3C Semantic Web begins to better support web users.

Semantic Web: Triple vs. Grand Slam

Sun, 30 Mar 2008 05:17:32 -0400

In this 5th post on the Semantic Web and our Web 2.0 inspired implementation, we are going to get to the heart of the <alt> meta data implementation.

In previous posts, we've looked at our divergence from the traditional Semantic Web and why it's necessary to do so: we have seen that user generated meta data is the only viable approach to implement the Semantic Web as publisher generated meta data is a model for control and is ripe for abuse. We've also demonstrated that user ownership of meta data is a requirement to make the Semantic Web a reality as only a user knows what he wants. And in the last entry, we showed that document-level meta data is irrelevant and users only care about the data contained within the document.

The <alt> meta data implementation has the following 7 characteristics:

Dynamic Meta Data. In the traditional Semantic Web, meta data is static as it is contained within the web page. As our meta data implementation is under user control, it may be easily modified as needed by the user or extended by a service provider. 
Versional Meta Data. Classically, meta data in the Semantic Web is not time stamped or versioned. In the Web 2.0 Mashup world, users need to know when was the last time a piece of web data was changed. As our meta data implementation is element-centric; versioning allows the users to see data changes over time enabling change detection and notification. 
Extensible Meta Data. In the most recent meta data standards such as RDFa, meta data is implemented in custom XHTML attributes. As it is hard-coded by the publisher, it is impossible to augment.   In our meta data implementation, for example, we encode an American English text-to-speech version of the HTML element thereby empowering the vision impaired. Users may extend the meta data with alternative languages and empower non-American English web users.  
Verifiable Meta Data. Publisher defined meta data whether in the form of embedded RDF, RDFa, or even Microformats may not be authoritative due to stale data-- usually due to page mirroring-- or forged sites. In our implementation, meta data can be compared by time stamp and other criterion to determine correctness.  
Executable Meta Data. In the traditional Semantic Web, most/all meta data implementations exist to provide data to RDF databases for analysis using languages such as SPARQL. In our implementation, the meta data may be used for query, verification, resource resolution, update, and application processing.   Speech synthesis is an example of direct meta data evaluation. 
Data Encapsulation in Meta Data. In the traditional Semantic Web, meta data does not contain the data it describes (or "repeats the data" as is described in RDFa standard.) Rather, it annotates the data.   As meta data is stored externally to the HTML page in our implementation: we store a copy of the original data, a text-to-speech version of the original data, and subsequent versions of the data. This approach allows multiple applications to process the meta data without fear of ad-hoc use conflicting with ontology standardization which usually requires several iterations to get right.   So, the <alt> meta data implementation supports both the needs of the many (users) as well as the needs of the few (ontologists)... the best of both worlds. 
P2P Meta Data Sharing. In the traditional Semantic Web, meta data is imported into an RDF database. A SPARQL query is then executed to obtain "knowledge".  This architecture only benefits service providers and precludes web users from getting the "knowledge". Just how many users are going to write SPARQL or install an RDF database? On the other hand, probably several "knowledge" service providers will provide a watered-down web interface in effect building walled gardens to control the user. Do you think that a service provider will allow user queries to be executed directly on "their" databases?  Our Web 2.0 tools and server support personal meta data registries where meta data can be browsed and shared. Today meta data is shared with other users manually, but we plan to support remote meta data sharing via peer-to-peer technologies.   A user can derive "knowledge" from 1 meta data source-- as we saw in the Wi-Fi coffee list example-- or through the composition of multiple meta data definitions. And as you should know, a data composition is also called a Mashup. 

So, now you've read about the 7 most important characteristics of our meta data implementation. And you also know why each one is a fundamental divergence from the traditional Semantic Web... saving the vision by updating the implementation.

Semantic Web: Documents vs. Elements

Fri, 28 Mar 2008 12:07:42 -0400

This is the 4th entry in our Semantic Web Series. By now, you know that we are shipping products and technologies to create a modern and pragmatic implementation of the Semantic Web. To accomplish this feat, we studied the Semantic Web vision and faithfully implemented a realistic solution which required a reinterpretation of some of the fundamental concepts.

In a previous post, I discussed meta data creation. In this entry, I will discuss our implementation of another fundamental concept: the URI... or IRI as it's now called. The <alt> Semantic Web implementation diverges in 3 fundamental issues as related to the URI concept:

Only URLs matter. In our implementation, we only care about page identifiers and not those mythical urn:isbn: URI/URN/IRI concepts. Why? Because we are generating meta data implicitly via the process of creating Mashups. So whenever a Mashup is created, meta data is generated.  Furthermore, URLs are important because a URL can be redirected, proxied, or otherwise intercepted. All other identifiers will not allow us to get the web resource. 
HTML elements matter and not the Document. As discussed in the post about meta data, the traditional Semantic Web defines a publisher centric view of the web. Consequently, the first item which is discussed as important meta data is "who is the author", "when was the document created", and other Dublin Core pieces. Who cares?   In these Web 2.0 days, the web has evolved to focus on the user. The Mashup user just cares about the data within the web page and not the dubious "creator". More concretely, as you can see in this 2003 screenshot of our Mashup tools, a user named Joe wants to repurpose a list of coffee shops into a mobile site. The user does not care about the ads surrounding the list. So Joe just wants the facts, just the facts.  So meta data must describe the HTML elements that are important to the user. And therefore the user should be able to create meta data to describe those elements which are of importance. Some elements might be important to some users but not other users. And that's OK because we empower users to manage their own meta data. And it's theirs because they posses it.  The Document is therefore just a shell containing data. Even though the Document is what is created; the original context of search and computation; and the unit of storage and transfer, it is superseded by the element in importance to the user. 
A URL provides context to elements and therefore meta data. Consequently it is more than an identifier. On the web, a URL is often used to resolve relative locations into absolute locations for image and link elements.  
So far in this series you have seen that we have made some fundamental changes to the classical Semantic Web with respect to meta data authoring and URIs. In my next blog post, I will write about our implementation of meta data.

Semantic Web: Revenues vs. Empowerment

Fri, 28 Mar 2008 03:40:59 -0400

This is the third post in the Semantic Web series. This entry is about the incentives to create meta data. For the publisher, the only/major benefit is to generate greater revenue. Consequently, no one on the web adds meta data unless they see dollars. On some web sites you are not allowed to unsubscribe because then your meta data can not be resold/repurposed. Your meta data represents "you", the person. So they have "you" even if you leave them.

That's why each and every credible Semantic Web community is unrelenting in its boasts of collecting user data as a means to generate revenue. A strong meta data implementation enables this far more than a normal customer database can.

Can we motivate web users to create meta data beyond their personal information?

We believe so. In the <alt> Semantic Web implementation, meta data creation is a by-product of Mashup creation. As users extract snippets of remote content, they build meta data which describes the content. This works because the data is important to them. Maybe not to me or you, but to the Mashup user it is.

Just like some people like certain online videos and others don't.

Here's a concrete example using our developer tools. Way back in 2003, when we starting building our Mashup tools, Sun ran this little overview of our tools. In this screen shot, the user decides to semantically extract a list of Wi-Fi coffee shops in San Jose.

Nowadays, a meta data file is created describing the list of coffee shops. In a future post, I'll describe how our meta data is executable and active unlike in the traditional Semantic Web.

But, back to the coffee shop list. This Mashup and consequently the meta data that is used is of benefit to San Jose Wi-Fi using, coffee drinkers. But not to the majority of web users.

Furthermore, we can see that ads take up most of the page real estate because the site publisher is an intermediary. Consequently, we cannot expect this publisher to create meta data devoid of ads. I have more trust in other Wi-Fi using, coffee drinkers. Or even better, I can author my own Mashup and manage my own meta data.

Our approach to enabling user-managed meta data parallels the real world in many ways. For example, just about every organization, group, or social network will provide their own list of "places of interest", directions, and facts. These are, by definition, mash-ups from various data sources customized to their particulars. They could just say "go map it" but most create or repurpose because they want to support their community, employees, and friends.

We believe the creation of Mashups is an adequate incentive to create meta data. And, since the meta data is automatically generated during Mashup creation-- there is limited additional work unlike in the traditional Semantic Web.

Semantic Web: Publishers vs. Consumers

Thu, 27 Mar 2008 23:26:52 -0400

This is the 2nd entry about the semantic web. The Semantic Web series compares the <alt> Semantic Web implementation with previous theories about how to implement the vision.

The authoring of meta data is a fundamental difference between the <alt> Semantic Web implementation and the Semantic Web standards.

Most likely due to the origins of the HTML, meta data in the traditional Semantic Web is to be created by the page author. Since she authored the page, she knows what's important. It is common knowledge that the META element has been abused to the point of irrelevance. So what about the newer attempts with embedded RDF or RDFa?

Same thing. You can't trust the page author to embed anything more than what generates ad dollars or installs malicious code. And, as the search engine vendors have definitively stated: "We deal with millions of Web masters who can't configure a server, can't write HTML. It's hard for them to go to the next step..."

It's actually even worse than that. As a tool vendor, we found out that even when XML is the only allowed markup-- as it is in the wireless world of WAP-- developers are not disciplined enough to learn the vocabulary/tags or even validate their content. This was a major reason for WML's demise.

Additionally, the newer Semantic Web standards such as RDFa or GRDDL are predicated on authoring XHTML documents. Two things should immediately jump out at you: first, XHTML is not supported by 80% of web browsers. And second, more than 99.9% of current web pages are not XHTML. They're written in HTML -- or broken XHTML-- and that's not going to stop for the foreseeable future.

The other problem is that many web tools and servers will mangle your perfectly formed XML/WML/XHTML. And finally, these extensions will cause your XHTML to not validate.

We believe that to implement the Semantic Web on an internet scale-- and not just in isolated domains-- we have to enable HTML content to to be used.

That precludes using XHTML and creator authored meta data. If it exist within the document and it's a credible site such as the BBC, etc well then it is one input of meta data.

So where do we get the meta data? In our implementation of the Semantic Web, we enable users to create, own, and share their meta data about a page. Just like users have successfully created and shared their own pictures, videos, and blogs.

And why not? If a reader doesn't find value in a page, then it's unimportant to them. But maybe someone else will find merit in a page. It is impossible for the author to divine everything that's important and then describe it as each user might want to consume it.

The only realistic solution is to provide users with the tools to manage their own meta data.

It's a Web 2.0 world: technologies should empower the user rather than providing greater control to the publisher.

Semantic Web: A Modern Implementation

Thu, 27 Mar 2008 22:17:04 -0400

As you should know by now, we provide Web 2.0 developer tools and server product lines. Specifically, we provide Mashup products and technologies which enable the semantic extraction, aggregation, and use of remote web site content.

You probably are asking: "Semantic"... why the qualifier?

Because we've augmented the underlying "HTML scraping" capability with various pieces of knowledge about the HTML and the data contained within the HTML. The sum of which provides semantic knowledge.

Our approach to page analysis is fundamentally different from the traditional Semantic Web. Why? Because the traditional Semantic Web approach to understanding web content is to require the page author to annotate the page with meta data and/or apply AI techniques such as machine learning to determine content relevance/knowledge.

Both approaches are unlikely to succeed. Others have discussed the failure of these approaches and consequently either dismissed the Semantic Web vision in whole or dramatically limited the vision to user data from social sites and "facts" from online sites such as Wikipedia.

[Begin 09 April 2008 Update]

Tim Berners-Lee suggested this correction:

"...The basic Semantic Web approach is to surface data which is already in data-oriented applications and relational databases using standards. it is not page annotation or machine learning! ..."

His comments and my response are below.

[End 09 April 2008 Update]

In this series on the Semantic Web, I will write about how we have built a modern and pragmatic implementation of the Semantic Web which is both faithful to the vision and achieves substantially more benefits to web users than previous designs or implementations.

The Redmond Comet: WebSlices Mashup

Wed, 26 Mar 2008 15:02:06 -0400

Following up on the blog entries about Microsoft IE 8's WebSlices and how to build a Mashup client; this post discusses the synergy between WebSlices and Mashups. It also discusses how to access a WebSlice-enabled Mashup.

Assuming you've read my previous post, you know that WebSlices implement a new type of Comet technology by shifting the programming responsibilities to content developers and having IE 8 directly manage all of the issues that would traditionally be handled through AJAX code: XHR, JavaScript timers, update functions, and notification.

As I mentioned in the blog entry how to build a Mashup client; a Mashup centers around the idea of extracting remote HTML snippets to create new content.

Furthermore to be useful, your Mashup has to support these additional items:

Refreshing the content based on some condition-- such as the Mashup server detecting a change in the remote snippet(s), a refresh based on a time interval, or a refresh based on the user manually requesting a content update.  WebSlices supports the later two types of content refresh techniques. The timer is client-based.   To instruct the Dynamic Mashup Server to encode the time interval in the WebSlices Microformat you should use the URL parameter refresh-incr. This corresponds to the ttl attribute of the WebSlice. An example URL is:  http://10.37.131.2?mode=webSlice-ie8&refresh-incr=1   The parameter mode=webSlice-ie8 is needed to instruct the Dynamic Mashup Server to wrap the Mashup content in the WebSlice Microformat.  
Inform the user when the last content update occurred.  When using other client technologies, you would use the URL parameter show-last-update to append the time of the content update.       As seen above, that's not needed with WebSlices since IE 8 will display the elapsed time since the last update. Just hover over the WebSlice in the Favorites Bar. 
Likewise, it is important to inform the user whenever the server is unreachable. WebSlices handles this well as seen here:        
Display the last content update even if the server is currently unavailable.   Once again, WebSlices does this well and allows you to manually attempt to open the original web page.     
Stop the content refresh. This can be encoded into the RSS update feed using the Microsoft extension element endtime.   To instruct the Dynamic Mashup Server to encode the termination time in the RSS feed, you should use the URL parameter ttl. An example URL is:  http://10.37.131.2?mode=webSlice-ie8&ttl=60  Likewise, the user can explicitly stop content updates through the Feed properties. 
Reconnect to the Mashup server. WebSlices also handle this as well by enabling manual refresh. It is not clear what is the automated reconnect policy. 
Easy access to the Mashup client. Opera Widgets, for example, allows a Widget to be top-level window.   WebSlices on the other hand are integrated into the Favorites Bar. So accessing a WebSlice is conceptually closer to accessing an Opera Panel than to an Opera Widget.  

So the complete URL to dynamically encode a Mashup as a WebSlice would be the following:

http://10.37.131.2:80?mode=webSlice-ie8&refresh-incr=1&ttl=60

and this is the WebSlice returned by the Dynamic Mashup Server

Now, let's grade WebSlices based on the checklist in how to build a Mashup client:

Access the Mashup server just like we would a normal web server using HTTP: no problem since WebSlices are browser based
The Mashup server can give us the remote HTML snippets as a whole new document or perhaps just as a document fragment using XHR. Don't forget, a remote snippet(s) can be transformed, filtered, and aggregated by the server as needed but conceptually it's still a remote snippet(s) of HTML: no problem since we are receiving a document fragment
The Mashup server can give us the mashed-up document or fragment as HTML, XML, RSS, or JSON: the WebSlice is HTML-based and updates uses RSS
The client has to be able to get updates either when the remote HTML snippets change or periodically when the user wants an update: WebSlices supports an interval encoded in the Microformat; a user defined value in the feed properties; and manual updates
The client can pull the updates using AJAX or the server can push the content updates using Comet: no problem since IE 8 will pull the RSS update feed
The client has to implement a strategy for handling HTML links to other content: IE 8 defines a policy for this
The client has to be easily accessible to the user to see or hear the content updates: no problem since WebSlices are integrated with the Favorites Bar though it doesn't seem that audio is supported in refreshed content
Distributing the client and handling software updates: no problem since IE 8 just reads the Microformat within the content

As you can see, IE 8 WebSlices make a great client for the Dynamic Mashup Server.

Microsoft Internet Explorer 8's WebSlices

Tue, 25 Mar 2008 21:24:53 -0400

This is the second review of Web 2.0 client technologies. The series started with this entry. Since our Mashup developer tools and the soon-to-be-released Dynamic Mashup Server support the Web 2.0 client technologies from vendors including Adobe, Apple, Google, Microsoft, Mozilla, Opera, Sun, and Yahoo!; I wanted to blog about the strength and weaknesses of these clients... in the context of Web 2.0.

So rather than the all too common puff pieces and hatchet jobs from less-than-qualified writers and bloggers, I'll write about our experiences providing commercial software in support of these vendors' offerings.

So, let's look at IE 8's WebSlices.

WebSlices has been developed by a group of innovative and highly skilled engineers. Leveraging industry standard technologies and techniques such as meta data, Microformats, RSS feeds, ATOM, JavaScript timers, and XHR; WebSlices introduces a new type of Comet technology.

Perhaps viewed as the next generation of Long Polling Comet, WebSlices adds a new temporal aspect to Long Polling... the termination time for the poll.

The innovations don't stop there. With WebSlices, Microsoft has shifted the architecture of Long Polling Comet from a client-centric technology to a more server-centric technology. With traditional Long Polling, the JavaScript browser programmer manages the connection, reconnection, and update logic programmatically. But with WebSlices, the server developer encodes the Long Polling parameters in a Microformat which IE 8 uses to perform the Long Polling.

The user is prompted to "subscribe" for content updates.

Content updates are automatically pulled by IE 8 either through an RSS feed or a special URL. In fact, a page can support several Long Polling directives.

So WebSlices introduces both an architectural shift as well a programmatic shift to Long Polling Comet.

So in the context of Web 2.0, here's what WebSlices gives you today:

An innovative use of server defined meta data to implement a Comet service
An architecturally-clean Comet technique
The use of non-proprietary technologies to implement a Comet technique
With the use of a Microformat, WebSlices is a competitor-friendly Comet technique
The dramatic reduction of writing AJAX code
Easy access to the WebSlice via browser Favorites
Persistence of the updated content and origin server information
Visualization of updated content even though the original web page has been closed

Here are some of the pitfalls of WebSlices:

Developers have to learn several advanced concepts and technologies including ATOM, RSS, and Microformats
Developers have to consider WebSlice CSS issues
Developers have to learn a new RSS namespace and element
Developers have to encode temporal data in W3C date format and not the more common RSS date format
Developers have to model and encode their content with temporal information rather than leaving this to JavaScript browser developers
Developers have to implement RSS feeds for content update
Developers have to provide multiple types of Comet techniques based on browser type
Refresh bugs or just inadequate documentation
Implemented in ehh... IE 8

From 1 to 5, with 5 being the best... here's the Web 2.0 client technology score card:

Vision: 4 out of 5
Use of Standard Technologies: 5 out of 5
Programming Ease for Server Developer: 3 out of 5
Unification of Server Developer Roles: 4 out of 5
Usable Today On Web: 1 out of 5
Usable Today In Companies: 4 out of 5

The New Face of Sun: Lively Kernel Mashup

Sun, 23 Mar 2008 20:24:22 -0400

Following up on the blog entries about Lively Kernel and how to build a Mashup client; this post is a tutorial of sorts on how to create a Mashup client using Lively Kernel.

Assuming you've read my previous post, you know that Lively Kernel already has support for AJAX and Comet. And, it probably has the best/easiest UI programming model because of its high level JavaScript API on top of SVG. But its achilles' heal is its lack of support for HTML, a missing layout engine, and multi-media support.

Don't forget, Lively Kernel is still a research project under development at Sun Labs and has only been publicly available for a few months. Nonetheless, Lively Kernel has the best chance of fundamentally changing the web experience for both user and programmer on both mobile and desktop devices... with a little help from friends.

So, we're going to follow the list in how to build a Mashup client and see where Lively Kernel stands.

First, we've created the code for a Mashup which is going to extract 2 snippets of HTML and aggregate them on the Mashup server. The first remote snippet is an HTML link to our Mashup sample data page. The next is just an HTML <TABLE> of some pictures... no, not the Sports Illustrated models. This time it's the next generation of Sun engineers and managers who are transforming Sun. Sun 2.0 if you will.

Some local content is added in the form of an HTML <BR> simply to separate the link from the table. Of course it could be whatever we want... perhaps content from Wikipedia on Dan Ingalls.

This is how the Mashup site looks in a browser:

and this is how the Mashup looks like in Lively Kernel:

When clicking the link in Lively Kernel, a new HTML page is opened and you leave the Lively Kernel environment.

So, let's grade Lively Kernel based on the how to build a Mashup client:

Access the Mashup server just like we would a normal web server using HTTP: no problem since Lively Kernel is browser based
The Mashup server can give us the remote HTML snippets as a whole new document or perhaps just as a document fragment using XHR. Don't forget, a remote snippet(s) can be transformed, filtered, and aggregated by the server as needed but conceptually it's still a remote snippet(s) of HTML: no problems since we used XHR to extract the document fragment and not the whole document since we only want the HTML elements contained by the <BODY>. That said, our rendering engine only supports a subset of HTML elements
The Mashup server can give us the mashed-up document or fragment as HTML, XML, RSS, or JSON: we used XML since the HTML layout engine works best by parsing well-formed HTML
The client has to be able to get updates either when the remote HTML snippets change or periodically when the user wants an update: no problem since we can use JavaScript timers or Prototype's Periodic AJAX
The client can pull the updates using AJAX or the server can push the content updates using Comet: no problem since we can use XHR, Prototype's AJAX, or Lively Kernel's NetRequest object
The client has to implement a strategy for handling HTML links to other content: our HTML rendering engine generates a click handler and will launch a new browser window
The client has to be easily accessible to the user to see or hear the content updates: Lively Kernel can be deployed as an Apple Dashboard widget -- and perhaps Opera Widget-- which will allow it to be a top-level window or remain in the widget layer. Likewise, Lively Kernel can be deployed in single-site-browsers (SSB) such as Mozilla Prism or Fluid.app so that allows desktop integration
Distributing the client and handling software updates: no problem since Lively Kernel and our HTML rendering and layout engine are JavaScript libraries

Since Lively Kernel is just a bunch of JavaScript libraries we need to ensure that they are bundled with the Mashup server. In version 7.3 of the XML Studio we do just that. It's found at:

your_install_dir/ALTMobile/objSvr/LivelyKernel-SourceCode-0.7/

We've created a SubWorld called Mashup World... which uses a hacked up version of the Network.js source code. You'll see that we use a NetRequest call to the Mashup server which is hardcoded to access a localhost server on port 33333.

So make sure that you start your Mashup on the correct port. And lastly, to access Lively Kernel you would enter the following URL in your browser:

http://localhost:33333/LivelyKernel-SourceCode-0.7/index.xhtml

So once Lively Kernel is friendly to 3rd party developers and you don't have to hack its source like I did... you can focus on using Lively Kernel's great graphics libraries and focus on your app.

So, what's a Mashup?

Sat, 22 Mar 2008 22:03:59 -0400

I've started this Web 2.0 client technology series to candidly discuss the real programming issues in the Web 2.0 world. Beyond the trivial "Hello, World", the irrelevant "Create A Circle With A Slider", or the juvenile game examples; I wanted to provide insight into what we needed to do to create Mashup clients on top of the industry's Web 2.0 client technologies from Apple, Microsoft, Google, Sun, Mozilla, Opera, Yahoo!, and the others.

Just in case you're not quite sure what a Mashup is: "It’s been called the essence of Web 2.0. It’s the ability to combine pieces of different web sites to create something new, something meaningful. Something for you and the people who have your tastes. Your social network. Not some mass market portal built by corporate programmers who think that they know you and your personal tastes.

Referred to as a composite web site by some and Mashup site by others, we call it amalgamating web data through the process of transcoding. Whatever. It’s about giving you the data that you want on your mobile phone or desktop browser. It’s Web 2.0. It’s about you."

A little more concretely, here's a screen shot of our Mashup developer tools-- from way back in 2003-- extracting an HTML <TABLE> of Wi-FI coffee houses and dynamically transforming that HTML element for use in a mobile site. So a Mashup is all about extracting remote HTML and inserting into a new site. That remote HTML can come from 1 or more sites, or web services, or databases-- from wherever-- and transforming them into different HTML as needed.

Of course, there are more details to this process such as fixing the incoming HTML or putting into a WML site but those are enhanced features built on top of the idea of extracting remote snippets to create something new...
copyright issues and DDOS aside.

Since we probably have the most knowledge of Web 2.0 environments-- and we want end users to be able to realize the potential of Web 2.0 and Web 3.0-- we feel that sharing our knowledge is the best thing for our company's financial health and to ensure that programmers are able to meet these new challenges and not get suckered into rewriting their old code for no good reason. Just how many clock widgets do we need?

So after blogging about each of the Web 2.0 client technologies, I'll follow-up with what's needed technically to create a Mashup client so that you can better measure just how complex your Web 2.0 project will be.

So first, let's black box the Mashup server piece and just conceptualize that it's a web server that knows how to extract snippets of HTML from other sites. Cool.

Now let's focus on what's needed to get that content from the web server using these newfangled Web 2.0 clients. You might be shocked :-(

We need to support the following:

Access the Mashup server just like we would a normal web server using HTTP
The Mashup server can give us the remote HTML snippets as a whole new document or perhaps just as a document fragment using XHR. Don't forget, a remote snippet(s) can be transformed, filtered, and aggregated by the server as needed but conceptually it's still a remote snippet(s) of HTML
The Mashup server can give us the mashed-up document or fragment as HTML, XML, RSS, or JSON
The client has to be able to get updates either when the remote HTML snippets change or periodically when the user wants an update
The client can pull the updates using AJAX or the server can push the content updates using Comet
The client has to implement a strategy for handling HTML links to other content
The client has to be easily accessible to the user to see or hear the content updates
Distributing the client and handling software updates

So those are the core requirements of a Mashup client.

We'll use this list to see how each of the Web 2.0 client technologies can handle them. With the goal being that you'll be better prepared for your Web 2.0 project.

Lively Kernel from Sun Labs

Sat, 22 Mar 2008 08:40:41 -0400

I am starting off the Web 2.0 client series with the most revolutionary of all Web 2.0 client technologies: Sun's Lively Kernel.

Lively Kernel is the culmination of the best aspects of Smalltalk and Self technologies and programming techniques: implemented in your browser with only JavaScript and SVG.

Said differently, the Lively Kernel team has created a JavaScript library which controls the native SVG that's built in Safari, Opera, and Firefox. With the upcoming Safari on iPhone and Opera Mobile browsers both supporting SVG, Lively Kernel just might become the unifying programming toolkit and runtime that works on all devices.

But wait! there's more... Lively Kernel uses the Prototype JavaScript library to do large scale meta-programming. So you get all of the reflective and dynamic code capabilities that possible with JavaScript.

No need for plug-ins or Java Applets. Just the browser.

[Don't tell the Java side of Sun or this project might get the kibosh]

So is that revolutionary or what? If you're a Web 1.0-type dude: no.

So in the context of Web 2.0, here's what Lively Kernel gives you today:

Solely browser-based so no need for plug-ins
AJAX and Comet support because it runs in your browser
Vector graphics controlled by high-level JavaScript APIs
A UI toolkit based on MVC and direct-manipulation
Built-in support for drag-and-drop and editing of all UI components
Built-in support for animation
Compatible with 3rd party JavaScript libraries and integrated with Prototype

Since Lively Kernel is a pure JavaScript environment, it needs to have a solution for the following issues:

Since Lively Kernel shuns HTML and uses its own JavaScript/SVG UI controls, more UI controls and better looking controls are needed. This was also a problem with Flash in its formative years
And another problem with shunning HTML is that all the world's data is in HTML. Fortunately, we've stepped up and solved that problem by providing a JavaScript library that parses HTML-- let's say from an AJAX call-- and then generates a corresponding Lively Kernel UI
Better support for multi-media objects using standard browser technologies. For example, the problem of sound seems to have been solved without HTML controls. Video needs a similar solution
An end-user mode where the UI is less editable. Something that is safe from erratic mouse gestures

From 1 to 5, with 5 being the best... here's the Web 2.0 client technology score card:

Vision: 5 out of 5
Programming Environment Setup: 1 out of 5
Programming Ease for JavaScript Developer: 4 out of 5
UI Controls Look and Feel: 1 out of 5
UI Controls Completeness: 1 out of 5
UI Controls Extensibility: 5 out of 5
Usable Today On Web: 1 out of 5
Usable Today In Companies (Demos Only): 3 out of 5

An Introduction to Web 2.0 Clients

Sat, 22 Mar 2008 06:50:50 -0400

Following yesterday's big release of our Web 2.0 developer tools; I wanted to write about our experiences building support for the various Web 2.0 technologies available to developers. If you downloaded our tools-- or just read the product documentation or viewed the screencasts-- you'll see that we support just about everyone in the Web 2.0 developer world: Apple, Microsoft, Google, Sun, Mozilla, Opera, Yahoo!... all of them.

In the next blog entries, I'll write about the strengths and weaknesses of the widget/gadget systems from Apple, Opera, Microsoft, and Google.

I'll also discuss the next generation of Microsoft's Internet Explorer and its WebSlices technology. And we will look at the Web 2.0 client technologies coming out of both Sun and Mozilla labs.

Not stopping there, I'll also discuss Adobe AIR and JavaFX Script. And since we've been on the forefront of Safari on iPhone development, I'll write about the real issues of mobile Web 2.0 support by Apple.

So if you want to get the real scoop on these technologies-- in the context of Web 2.0-- come back.

Announcing XML Studio v7.3

Fri, 21 Mar 2008 07:12:35 -0400

We are pleased to announce that version 7.3 of the <alt> XML Studio is now shipping. Featuring the industry's first commercial Comet developer tools, the XML Studio breaks new ground in the development and testing of Web 2.0 applications.

Developers can now build and test real-time streaming mash-up applications. Version 7.3 embeds a copy of the upcoming <alt> Dynamic Mashup Server which uses a custom version of Grizzly, the Comet technology from Sun.

In addition to its Web 2.0 server support, version 7.3 of the XML Studio adds mash-up developer support for 3 new Web 2.0 client technologies: Mozilla Prism (the single-site browser (SSB) technology from Mozilla Labs), Sun's Lively Kernel (a JavaScript/SVG web programming environment from Sun Labs), and Microsoft's WebSlices (a new feature of Internet Explorer 8).

Previous versions of the XML Studio introduced mash-up developer tools for Opera Widgets, Apple Dashboard Widgets, Adobe AIR, Microsoft Vista Sidebar Gadgets, RSS mash-up feeds, Yahoo! Pipes, Google Gadgets, JavaFX Script, and Safari on iPhone.

Version 7.3 of the XML Studio also adds new capabilities to its Web 3.0 semantic web developer tools. Now, our RDF mash-up meta data includes support for SSML-- the W3C standard for speech synthesis. By explicitly including text-to-speech technology in our mash-up RDF definitions, we hope that both the visually impaired and non-English speaking users will become full participants in the mash-up revolution.

The <alt> XML Studio is available for Mac OS X, Windows XP, Windows Vista, Red Hat Enterprise Linux, SUSE, Ubuntu, and Solaris.

To download a copy of the <alt> XML Studio, visit http://altmobile.com/Home.html

Announcing the Dynamic Mashup Server

Tue, 5 Feb 2008 09:01:04 -0500

We are pleased to announce the <alt> Dynamic Mashup Server. Built to support internet scale amalgamation of web content, the Dynamic Mashup Server is the industry’s first Comet enabled mash-up server.

Implementing all of the major AJAX and Comet technologies such as XHR multipart, forever frame, XHR streaming, and forever script; the Dynamic Mashup Server supports HTTP Streaming to desktop browsers as well as Safari on iPhone. Single-site browsers (SSB) such as Mozilla's Prism and widget platforms such as Opera Widgets, Microsoft Vista Sidebar Gadgets, and Mac OS X Dashboard are also supported clients.

Comet services are provided through a custom version of Sun's Grizzly web server technology.

Other cool features of the Dynamic Mashup Server include RSS mash-up feed generation and text-to-speech output to support visually impaired user access.

Implemented to run on 64-bit Java Virtual Machines, the Dynamic Mashup Server integrates into our virtual Web 2.0 containers which provide isolated access and execution of remote web content ensuring that mash-up clients are protected from potentially harmful HTML and JavaScript. Mash-up isolation is also necessary to avoid XSS attacks.

The perfect companion to our Web 2.0 developer tools, the Dynamic Mashup Server supports mash-up RDF meta models which provide a realistic implementation of the semantic web.

The <alt> Dynamic Mashup Server will be available in Q2 of this year.