Posts Tagged ‘computational semiotics’

Albert Einstein wrote: “Computers are incredibly fast, accurate, and stupid. Human beings are incredibly slow, inaccurate, and brilliant. Together they are powerful beyond imagination.”

The partnership between human beings and computers is long and enduring and there are so many examples of just how powerful the influence of computers really is. This was especially true after the debut of the personal computer, and again after the debut of the Internet that gets us connected today.

When spreadsheets came out we became better tabulators. When word-processing and spell-checkers arrived we became better writers. The widespread use of relational databases made it easier to collect, store and manage information making us more intelligent about larger collections of data.

Over the decades of computing the costs of storing data have dropped to nearly nothing.  In many cases storing data on the Internet is free.  The costs of collecting data has dropped significantly.  There was a time, not so long ago, that the 300 baud modem was the most common way to connect or be “on-line” with another computer.  The costs to download 10 megabytes over long distance telephone lines was not inexpensive.  Now people connect to the Internet over public wireless networks in most cities. It is offered free by many business establishments. People now download a thousand times the amount of data moved in 1985.

But something went wrong. The five basic means and capabilities needed for intelligence are collection, storage, retrieval, analysis and dissemination. We have systems of collection, storage, retrieval and dissemination but the systems we do have for analysis are not generally something anyone can run on their personal computer.  Even if we can run them on a desktop pc, they are complex systems that require significant expertise to make them work well in limited areas of specialization.

Analyzing the patterns and ordering the data helps us learn about the world and obtain to better and more complete theories.  Albert Einstien wrote:  “Concepts that have proven useful in ordering things easily achieve authority over us that we forget their earthy origins and accept them as unalterable givens.  Thus they might come to be stamped as “necessities of thought,” “a priori givens,” etc.  The path of scientific progress is often made impassable for a long time by such errors.  Therefore it is by no means an idle game if we become practiced in analyzing long-held  commonplace concepts and showing the circumstances on which their justification and usefulness depend, and how they have grown up,  individually, out of the givens of experience.  Thus, their excessive authority will be broken.  They will be removed if they cannot be properly legitimated, corrected if their correlation with  given things be far too superfluous, or replaced if a new system can be established that we prefer for any reason.”

Yet, still, here and now as we are in the twenty-first century we are lacking knowledge of those things that are given in our individual, private, and our public, social experience.  There is no model, no theory by which we can know, count and measure the givens of experience.  Einstein also wrote that: “It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple as possible without having to surrender the adequate representation of a single datum of experience.”

So, it is a fair question to ask after the adequate representation to the givens of experience.  It is reported that in a letter to his son, Einstein wrote that: “Life is like riding a bicycle.  To keep your balance you must keep moving.”

Isn’t it time to move on to a new way of thinking about intelligence and our means and capability to alter the structure and order of our independent, yet collective reality?  This video below defines simple basic and abstract elements of thinking that could make it possible for computers to do more intelligent analysis in much simpler ways, and to help us become better thinkers in the process.

Read Full Post »

For all those just joining me on this multi-part post, in this part I will write about how we derived computational objects by abstracting them from the significant or semantic properties of being a human in this world.

I will also introduce you to the notion that the sounds of natural language indicate and interpret compound objects representing all possible actions or (human) processes, and the boundary and engagement conditions of personal action and interaction. Besides that, throughout this piece, I will also note where this sort of theoretical model fits and compares with modern computer technologies. So that I can give all these aspects proper consideration, this part will be slightly longer than the previous two parts.

We want to model interpersonal relations on computers because computer software does not relate to the environment of people or the world that we inhabit very well if at all. Now that the search engine is a common appliance, more and more people are realizing that despite decades of costly research, computers are nearly clueless of the implications of words and common expressions.

It goes beyond search engines to computational semiotics. A lot of people have trouble interfacing with computers. I believe this is because there is no framework for computers to understand people. People have empathy towards each other. What do computers have? Maybe if the computer were more understanding, we would have better computer interfaces.

If computers are ever going to “understand” people better it makes sense to start by identifying with people and their environment. Understanding something or someone always begins by identifying with the situation. Most computer software does not do that; data is generally a dry and often drastic reduction of the situation. Natural language, for instance, is reduced to its terms and their parts of speech.

Term vectors in text processing systems are statistics of the occurrence of terms in documents and in collections of documents. They are a representation of text that the computer simply and blindly records and accounts for without any consideration for any other relationships beyond the needs of the application. This is not very intelligent. The term “computational semiotics” intuitively suggests that we use a science of meaning to create useful and intelligent computer programs.

One place computers are trying to achieve more intelligent processing is in text and document analysis. Here, in this and related fields of Information Retrieval (IR), a “better understanding” is measured according to recall, precision and relevance. Partly due to the growth of web pages on the Internet, a major focus of advance systems development is on Natural Language Processing (NLP) and text search and retrieval, or the search engine.

In modern indexing and computer classification systems, terms are extracted and term vectors are considered using local information from a text (its terms and sometimes its structure) and “global” information from collections of texts or documents. Considering that most texts are about interrelationships between people and things or principles in the world, it seems modern computer systems may be missing something.

Consider the interrelationships of any text to everyday affairs. Consider Aesop’s fables, and then consider how most of the text on the Internet is, at its root, about the interrelationships between people and their environment. This is the problem with term vector schemes. The signs are not interrelated to everyday affairs in the world and this is why there are limits to these computer methods and systems.

If I know what to look for on the Internet, I can look up documents, articles and references, about any person or event. Yet even though I am an expert, the experience is not always satisfying. I can ask my wife to watch out for print articles and news that will interest me as she is an avid reader. I don’t have to specify the structural relationship or the Boolean logic or look up any vocabulary for her to use. Obviously computer software cannot do the kind of intelligent processing that my wife can. This is why I am producing Readware: a semantic framework for software engineering of text analysis, classification, search and retrieval applications.

Readware can identify the interrelationships in a text using a call to its API:

rwAnalyze Aesops fable

In the image above, you can see the input string on the left and you can see that the output is a ranked listing of the interrelated topics. The top five categories and topics characterize the categorical implications of this text. The topics are representative of everyday affairs. Readware categories and topics can be used as filing, filtering and routing options in a computer application. Some of them repeat because the topics come from different perspectives.

My company has theory and logic for interrelating expressions on web pages and in messages. We developed our theory and logic into a platform with a well-tested API for use on various sorts of content. We developed our computational solution at about the same time people were thinking of WordNet and the Internet and before the Semantic Web.

Theory is harder to adopt than standards in the modern engineering communities of software developers and computer programmers. The Semantic Web has enjoyed tremendous support. Their philosophy also follows in the traditional practices of AI where you identify all the special properties and relations of everything in your content. The only difference with the Semantic Web is that those folks want you to use web formats and standards for your data that put it in a form that is provable using the various implements and standards endorsed by the W3C (OWL, RDF, XML, etc.).

So that I am clear, there is no theory of semantics, interpersonal, or otherwise, in the confines of the Semantic Web. Nowhere in any of the published standards of the W3C will you find any hint of the properties and relations of interpersonal semantics. Most AI theory has been focused on NLP methods from mainstream linguistics along with functional grammars that have been around since the 1960’s. Not that it is bad, they have been at it a little longer than Tom Adi and myself. I just mean that none of these well-funded systems have enjoyed ultimate success; otherwise this topic would be moot.

Google and several other companies have risen to prominence using search engine technology that indexes the full text of documents, articles and web pages. They showed that you don’t need expensive language processing and AI methods. Google supplemented the text with a quickly calculated popularity measure. This measure made it possible for Google’s search engine to provide the most popular links in their results. However, if your query is the least bit ambiguous it will not matter how popular the sites in the results are because they will be more or less irrelevant.

Besides Google, the recently arrived alternative technology to the Semantic Web are highly refined (and very expensive) NLP systems, and “semantic search engines” represented by great organizations, such as Hakia and PowerSet. You can see for yourself that these NLP systems do not “understand” much in the way of the world. While they are called “natural language systems” you cannot have a conversation with them. If you could, you would find that they have very limited knowledge of the way things really fit together.

This effect can be seen at the new Semantic Web based online search service for news and current events called Silobreaker, where they claim:

It recognises people, companies, topics, places and keywords; understands how they relate to each other in the news flow, and puts them in context for the user.

I tested that out on Tuesday, 12 February 2008. I clicked on a link to Presidential candidate Barrack Obama, from the Silobreaker front page. I was expecting to be shown something about Barrack Obama. Instead I was presented with a network graph:

Valid Relationships ??
It reportedly drew a relationship between Barack Obama and Justin Timberlake in the context they identified as a story from the Denver Post entitled “West, Winehouse take early Grammy lead”. It did the same for the other artists listed in the graph.

Frankly, it is just too much information for me. In this instance, it is adding to information overload because there really isn’t any relationship there. The relationship, if any, is between the Grammy Awards and the people mentioned. There is no relationship between Barack Obama and Justin Timberlake. It is wrong to infer otherwise from the context.

Part of the problem here is that the Grammy Awards are not a known entity in this static entity relationship model. The problem stems from the lack of comprehension of the interpersonal relationships involved in the context.

Neither the Semantic Web, nor AI, nor natural language processing can solve this problem. The problem is that natural language understanding is too naive and the truth-based semantics are too superficial. Yet, scientists have been reluctant to delve too deeply into the abyss of personal psychology and interpersonal beliefs on the basis that such beliefs are illogical and unscientific.

We did not buy that argument then or now. The basis or ground rules of interpersonal relationships are often characterized as psychotic and pathological, religious zealotry or astrological nonsense, although more concrete grounds have existed. No one thought of abstracting from the possibility of personal action devoid of any other notions.

That is how we are born, devoid of belief, character, nationality, philosophy or religion. It is only after grasping sounds and a language that people start filling their minds with these notions. While I have been subject to these dubious notions, Tom Adi drew us both to elements abstracted from existence and action. We found them right there, literally in front of our face, and in the smallest elements of meaning in natural language.

In part 1, I began by presenting my experience with computer systems and how we began looking for natural systems and their semantics. In part 2, I identified two indispensable sets of universal properties represented by being human: 1) a body-centered reference, and: 2) power. I trust my reader perceives these properties as self-evident.

My claim is that these properties are an affordance of individual influence and worldview. Recall in parts 1 and 2, I explained that a major premise of this work is that the elements and relations of natural languages should correspond with (the elements and relations of) other systems of natural phenomena at all times.

In the original research that I cited in part 2, Dr. Tom Adi found a natural correspondence between elements of natural language systems (sounds/phonemes) and the abstract objects used in individual cognition or recognition. I am going to expand on his finding and show how the semantics he proposed correspond with the actions and interactions of people.

The fundamental elements of language are its signs. Linguists have told us that words are signs; so are names. These signs are linguistic signs that reflect elements of social conventions and elements of design and even some modicum of chance. The same signs also reflect the power and influence of the individual voice and imagination. The fact that this power is latent in words and in text means that it is present and accessible in the unconscious mind but not consciously expressed.

This supports my charge that something is missing when all that we are left with is an alphabetical index of keywords. Using NLP on sentences of a text to decompose it into nouns, verbs and other parts of speech does not capture or address the power and influence afforded by the author, and therefore, computer processed text looses its luster, it looses its possibility for action and it looses its capacity to influence the reader.

This is another reason the linguistic, NLP or AI-like approaches to the problem of understanding the functional role of language do not work very well. When they parse a text into sentences and a bag of words, they leave out the import of words. By that I mean that modern text processing methods miss the part that carries or holds the meaning– the influential part — the significant part.

Hidden in the smallest particles of meaning, the phoneme, are elements of action (power) and interaction, these are abstract signs of the conditions existing in social interactions. They are abstractions of the boundary and engagement conditions afforded by our body-centered reference. The interpretation of meaning appears to flow from the axis of the abstractions of power and those of the perceived boundary and engagement conditions.

In psychological studies of the interpersonal relationships between people, psychologists agree that people interact implicitly and explicitly. People can be focused on the interaction, as in a sales situation, or unfocused and interacting, as people together on a bus going to the same ball game, for example. I do not want to take you from here all the way to social psychology and symbolic interactionism though both of these fields are related to what we are talking about here because language is our main tool for socialization.

I will borrow terms and polarities from social psychology and interpersonal relations, such as implicit/explicit, focused/unfocused, and open/closed, to explain the boundary and engagement conditions. In “Society as Symbolic Interaction” (1962): Herbert Blumer claimed that people interact with each other by interpreting or defining each other’s actions instead of merely reacting to each other’s actions.

Blumer wrote that the response of people is not made directly to the actions of one another but instead is based on the meaning which they attach to such actions. We do not disagree that human interaction is mediated by the use of symbols and signification, by interpretation, or by ascertaining the meaning of one another’s actions.

Interaction is always (at the very least) bipolar. Because interaction is a fundamental pillar of group dynamics it should come as no surprise that polarity is a feature of the language used by the group. We found that polarity is represented by compound abstract objects related to boundary and engagement conditions. In the book Semiotics and Intelligent Systems Development, Tom Adi wrote:

We believe that the phonemes of a word are signs that refer to abstract objects that are somehow related to the properties of the object to which the word refers:

    word X refers to object A
    each phoneme P of word X refers to an abstract object B
    abstract object B is related to property T of object A

Moreover, we believe that the human mind constantly interprets such abstract objects and that the resulting interpretations also can be abstract objects that may in turn be reinterpreted. Both the original abstract objects and their successive interpretations are related to the properties of the object to which the word refers.

    abstract object BP is interpreted as abstract object B’P
    abstract object B’ is related to property T’ of object A

In addition, we believe that the morphology of a word, its structure, is also a sign that refers to an abstract object structure that is somehow related to the structure of the object to which the word refers. The human mind also constantly interprets and reinterprets this abstract object structure.

    structure of word X refers to an abstract object structure S
    abstract object structure S is related to structural property T of object A
    abstract object structure S is interpreted as abstract object structure S’
    abstract object structure S’ is related to structural property T of object A

The repeated interpretation of the abstract objects to which the phonemes of a word refer, in light of the repeated interpretation of the abstract structure to which the morphology of that word refers, will establish more and more relationships in the human mind to the properties of the object to which that word refers. We call this principle cognitive growth by reinterpretation.

A similar growth by reinterpretation is found in biosemiotics, the study of DNA as signs of life processes. In the book “Signs of Meaning in the Universe“, Jesper Hoffmeyer wrote that Repeated DNA interpretation produces biological growth along a path called the ontogenetic trajectory. This parallel is not surprising since human cognition is born of human life processes. We expected to find relations of symmetry between the abstract objects to which phonemes refer since language is a natural phenomenon and there usually is symmetry in nature.

In Tom Adi’s study of the Arabic language he found that certain sounds interpret a compound abstract object he named closed and self as an interpretation of the influence on his mind. Certain sounds interpreted the compound objects open and self to his mind. He arranged these sounds in symmetrical columns and found that the sounds in these two columns expressed a kind of polarity, he called inward and outward.

He expected to find similar abstract objects interpreted by the remaining sounds and he found the abstract objects, closed, others and open, others. And he arranged those sounds according to the polarity they expressed, that of, focused interface, and its inverse, unfocused interface.

In English, sounds are far more ambiguous, yet they can be arranged in a similar fashion. By examining small words with a single consonant or sound we can see how the sounds of English interpret the same abstract objects. The personal pronoun “I” indicates the bipolar object: closed, self. This expression (I) directs attention inward with the focus on self. Its counterpart, the personal pronoun “You” indicates the symmetrical compound object: open, others. Our attention is directed outward and the other is targeted.

How about the pronouns “We” and “Us”. These interpret the compound object: closed, interface. Our attention is directed to a collective of objects or entities as in ‘bringing us together’. Its symmetrical counterpart: open, interface, is indicated by the personal pronoun “He”. It is not neither of us but a third person, him. It is the symmetrical opposite of we and us, it is they and them.

Now, with the principle of cognitive growth by reinterpretation in mind, as I mentioned above, try to imagine yourself a small child before acquiring language, if you can. Imagine how often you might hear just these sounds and the situations in which they are used. Imagine how often you would interpret these sounds, at first tentatively to be sure, and then with more confidence, as the learning became ingrained in your mind and in the neurons of the brain.

It seems that every phoneme, every element of meaning, in a natural language interprets a single compound abstract object, in the same way as I have shown for the personal pronouns of English. These abstract objects are called compound because each is composed of the abstraction of a definite set of boundary and engagement conditions.

Most consonantal sounds of a natural language also indicate an abstraction of power. This is an action predicate defining specific sorts of processes. Action may appear to just be, while it is really built up; it is a construction. According to Blumer and his teacher Mead, action is built up step-by-step through self-indication. Tom Adi recognized the objects he found as basic categories of abstraction. This is a set of elementary categories for all types of identification, all types of manifestation, and all types of ordering.

Identification deals with identities The sounds of words dealing with identities are who, which, I, you, we, he, it and units and elements one, a, an (the bold letters emphasize the chief sound). Those are static interpretations. Dynamic interpretations include assignments (at, and, a-) and existence (is, are, on, off, at).

Manifestation is the way things present or manifest themselves. The sounds of these words indicate this abstraction: Matter, mass, medium, field, pool, form, domain and theme are examples of static manifestation. Motion, formation, phase, application and doing exemplify dynamic manifestation.

Ordering is expressed by the sounds of the English words: numbers, names, quality, quantity, quick, quadrate, quarter, as well as energy and force (knock, quake, quench, quell), awareness , perception and feeling (notice, qualm, numb), sound (ring, quiet) and cognition (know). The negative meaning of no, non-, un-, in-, etc. comes from the bipole attached to n. In English, negation is an organizing act.

These elementary categories expand into a power set of eight categories that are represented by all the sounds/phonemes in the language.  Many consonantal sounds form bonds (inside a word) that interpret combined actions or processes.

I see this piece has gotten rather long. My aim was not to make a complete presentation but to highlight some of the features of the approach and intelligent computer software I am producing. I hope that I have clarified that personal action and the dynamic boundary and engagement conditions relevant to every situation give way to the interpretation of meaning from language. If I get to another post on this subject, it will explore the bonds of abstract objects inside words and how meaning is composed in ways similar to how action is constructed.

As always, feel free to leave me a comment by clicking the link to comments below.

Read Full Post »