LjSEEK.COM: News and Announcements

MyRating.ljseek.com - know your rating!

Thu, 20 Apr 2006 05:41:50 GMT

We're happy to announce new addition to ljseek.com family.
Myrating.ljseek.com is a tool allowing to track your LiveJournal
link popularity. Here is example how it looks:

My rating

Find out more at myrating.ljseek.com

Winter news

Tue, 17 Jan 2006 08:28:11 GMT

About 3 months have passed since our last news update. We've
implemented number of new features in this time and started few
new projects. Here is the short list:

Fine grain control over indexing and caching
Browse LiveJournal content by tags
Find Journals by their name and description
Image Search.

Fine grain control over indexing and caching

We've been getting number of requests to remove posts which were made
friend only or deleted, avoid indexing certain posts or index journal
even if it is not allowed to be indexed by major search engines. Now
you can do all of this.

You can register with LjSEEK and add your journals to your account
(we'll perform validation control at this stage to ensure existing
rights for this journal) and specify indexing options as well as edit
our database to remove specific posts. There is also tiny Ajax
application available which allows to automatically remove all deleted
and entries as well as entries made friend only.

Browse LiveJournal content by tags

www.LjTags.com allows you to browse LiveJournal content by tags. Same
as you can browse your own post by tags now you can do with posts in all
journals. We also compute related tags and have graphic functionality
which may be fun to use, for example
http://halloween.ljtags.com/graphics.php?tn=halloween%
2Cchristmas&period=1&average=sma_7 will show you how big holidays are
Christmas and Halloween in terms of people writing about them.

Find Journals by their name and description

Now you can find journals, communities or both by their description at
journals.ljseek.com. If you feel like reading something fun try
searching "humor"

For each journal we'll also display the last post snippet so you can see
what is it really about.

Image Search

www.LjPIC.com now not only provides you with thumbnails of recently
posted images, but it also has search functionality to find images on
specific topic. We now have more than 15.000.000 of images in our
database. As with LjSEEK you can limit search to certain users and
their friends. You also can browse picture previews for specific user:
http://www.ljpic.com/users/photographers/

LjSEEK.COM: September News

Tue, 11 Oct 2005 05:58:13 GMT

Huh, We're late with update this month, also instead of description
of new nice features we've implemented you've got downtime warning in
the start of October.

Now there are finally some good news.

First we're pleased to say stability problems seems to disappear the
same mysterious way as they came. The box is not crashing everytime we
start to update index any more while we were told no hardware was replaced
or anything.

Second - we're pleased to announce we're moving this service from rental
servers to our own collocation faculty in San Jose. At the same time
we're doing software updates - this version is up and running at
http://beta.ljseek.com - give it a try and let us know the problems
you'll find out. We should point main site to the same location soon.

From the glance view Beta version looks very similar to old one.
There are however serious changes done in underlying software.

We have updated our database structure so we'll be able to add more
features soon as well as get some speedup.

We've moved to utf8 so now non-English language search should work
better.

We've improved thumbnail generation so images are now better quality.

Setting up the collocation took a while - getting hardware, setting it
up. working around some bugs took a while. We moved our system to
x86_64 platform (64bit Xeons) and had to do some minor porting to make
programs written in C work on this platform.

Unpleasant surprise was to see PHP code also need some fixes. For
example crc32() function started to return unsigned integer instead of
signed breaking a lot of stuff. Eaccelerator which we used for caching
and locking breaks on 64bit architecture so we had to move to APC.

So if you're interested what this site will be running now it is Dell
PowerEdge sc1425 for Web Box with 2 3Ghz CPUs and 2GB RAM and Dell
PowerEdge 2850 with 8GB of RAM and 6 SCSI drives in RAID10 as Database
and Search Index box.

In the months to come we're going to add more hardware to offset search
index growth and traffic increase.

LjSEEK.COM Search service experiencing downtime.

Sun, 02 Oct 2005 05:31:47 GMT

Our search service is temporary unavailable. We're working on problem
resolution and will try to fix it as soon as possible.

Technical Details:

Our main search index server is having hardware problems right now - it
just powers off itself during times of high load. Unfortunately our
hosting provider was yet unable to resolve the problem.

LjSEEK.COM and Privacy

Fri, 23 Sep 2005 03:01:09 GMT

Recently we've been getting lot of questions and requests regarding
privacy. The privacy is extremely important to us. Usually journal owners appreciate
their posts to be searchable, which increases journal audience allows to
find more friends and get more comments, some however would like to limit
visibility. But how do you discover what journal owner really wants ?

This is tough question to solve - if you take a look at search engines -
these would index any page they find on the Internet, unless special
tags are used or special file (robots.txt) created to forbid them doing
so.

In RSS, which we use for indexing, there are no such tags, so many blog
search engines would simply index everything. As we only index LiveJournal
journals we can do it better - we check if user had set "Minimize your journal's inclusion in search
engine results" and do not index such journals. This method is however
not 100% reliable. First - we can't catch changes in this setting at once
- due to large number of Users at LiveJournal we only check this
flag when we add new journal to index and about once per month. You also can use Journal Tools Page
to update it. This way
also does not work with some custom LiveJournal page designs.

Indexing Friend-Only or Private Entries.

At this point we simply do not do it
and we even could not do it if we'd like. LiveJournal protects your data
reliably at our best knowledge. We would have to ask your password to be able
to index private and friends only posts. If you find some private or friend only
data in the journal this must be data which was public and only later
was made private. We're using RSS interface to get information updates
and we simply have no way to discover if some post become private/friend
only. If this is a concern for you drop us a note, listing which posts
you need to be removed and we'll try to help you.

Information re-publishing.

We always try to follow fair-use
principle when re-publishing information. In search results we only
display snippets so users are directed to your journal for full posts.
We also have "Cached" version page which show full post, which is done
for convenience purposes allowing us to show version which we have in
the database (which might be updated since then) as well as highlight
matching terms. This is quite similar to what Major search engines do
with web site pages. We also do not show any advertisements on cached
page.

Future Improvements.

We'll soon
add user registration feature which would allow users upon registration
and confirmation of ownership of certain journal allow to control its
state in our database. One would be able to enable/disable indexing
independent from global LiveJournal settings, block display of cached
version or snippets, remove posts which you would not like to be
visible. Stay tuned.

LjSEEK Plugin for FireFox is now available !

Mon, 19 Sep 2005 05:38:04 GMT

aelana has created LjSEEK Plugin for Mozilla/Firefox
You can
Download it here

This is truly great contribution !

If you have any development suggestions or have discovered any bugs
please let us know

We must note however this plugin was was created and maintained by
third party so we can't be responsible for its functionality.

Mon, 29 Aug 2005 05:29:18 GMT

The month has passed since we write any news. We have not been lazy, on
the contrary we have been adding features so did not have
time to tell about them.

Main Changes:

Images from posts are shown in snippets (example)
URLs are now indexed in posts
Search results are available in RSS format (example)
Posts Linking to given journal are visible (example)
See which posts are indexed (example)
-Strict/Normal/Relaxed match modes
New Service http://www.ljpic.com

Images from posts are shown in snippets

So you can get better understanding what this post is about without
clicking on it. We retrieve and cache thumbnails on our servers so it
will not be eating up your image server traffic. We spot images which
are shown in the post as well as linked images.

URLs are now indexed in posts

Previously we were stripping all HTML tags from post before indexing it,
now we have left some tags, mainly containing urls as well as link and
image descriptions. This should greatly improve search results if you're
looking for some web sites. For example number of matches for "ljseek"
tripled by enabling this feature.

Search results are available in RSS format

That says it all, enjoy. We would however encourage you to refresh this
channel no more than few times a day (at this point indexes are updated
once per day anyway) to avoid overloading our servers. You may also
select ordering by time in RSS format so you see new matches in the
order they appeared in LiveJournal.

Posts Linking to given journal are visible

In one of previous updates we've added star rating - tiny stars under
user/community name, which reflects number of inbound links to this
journal. Now you can click on the star to browse them. The number of
links on the star may be different compared to one when you click on it,
as we only update star rating information once per week. This data is
also available in RSS format so you can easily track who is linking to
your journal.

See which posts are indexed

Now you can see which posts we have in our database for each journal,
this way you know what is indexed and what is not.

Strict/Normal/Relaxed match modes

By default matching is done in Normal mode. Strict mode is the same as
exact phrase checkbox - we just made it simple to switch to it, the
Relaxed mode does not require all keywords to be in the posts, but any
of them. Relevance ranking will however rank posts with largest number
of matches first.

http://www.ljpic.com

This is a toy which we've added while debugging thumbnail generation and
caching. It is fun to play with - you can see last pictures posted to
LiveJournal, which are updated every few seconds. It also allows you to
take a look at random pictures if you find nothing interesting in new
ones. You may think it is same as many similar scripts available in the
Internet, however it is not. We do not link full pictures but
pre-generate thumbnails which means images load very fast from our
servers and consume very little traffic. Actually this service is even
usable with dialup connection.

Development news update.

Thu, 28 Jul 2005 04:54:20 GMT

We're continuing working hard adding new features and improving old
ones. Last two weeks passed as productive and rewarding we got a lot of
feedback and questions from users so we could extend our FAQ section.
We're however hungry for more. Tell us what you like in our service,
what you do not and what would you like to see.

Exciting news came from LiveJournal.COM - LjSEEK.COM search service is
now used to provide users search withing their own journals. Go to user
profile (example <http://www.livejournal.com/userinfo.bml?user=ljseek>
and click on magnifying glass. This should make access to our service
much more convenient for LiveJournal users looking things withing their
own journal. If you would like to be able to have quick access to
global search function you can still use our personal search form

A lot improvements to navigation were implemented. Now we'll show
below search form which filters are in place now and save settings from
advanced search for improved search experience.
We've added "Journal Status Page" which allows you to check status
of your journal and add or remove it from the index at your will.
Now we're indexing over 30.000 communities and the number is still
growing. If yours is not in the list add it. Note communities
frequently have a lot of traffic so the history might not go that far.
However all new posts are added to the index which makes will results in
great archive in a months
Cached posts previews are now available. These will show full post
with highlighted found search terms.
Time Separators in search results. You may see "Last Week" or "Last
Month" in search results - these are to visually show how old
information are you looking at, when sorting is done by time or time and
relevance.
"Star Signs" - Now you can see these under user picture for some users
and communities. if you put your mouse over them you'll see number of
incoming links we have to this user journal and from how many users are
they. "Rating" is your place by number of incoming links. They may look
low as place is shared by many users having the same number of incoming
links. What is it for ? Just for fun and convenience. We're not using
this data for search result sorting.
Added new bugs. Yes I bet so. New features can't come without bugs.
Please do not leave them alone let us know so we'll take care of them.

LjSEEK.COM: Development News Update

Tue, 12 Jul 2005 06:07:27 GMT

During last week we worked hard to add few nice features to our service.
- Other 20.000 communities are now being indexed. We're still adding
few thousands per day. Once link to community is found in indexed
journal it is set for indexing, so you should not do anything special
to add your community to the index.

- We've added "relaxed search mode", which will find posts even if they
do not contain all keywords from search phrase. It will be automatically
offered to you if normal search matches no documents (example)

- We now catch spelling errors and offer you to search corrected phrase.
It is not as smart as Google's advices but still well catches most
typical spelling errors (example)

- Finally we've created personal search form, which you can place on
your website, add to memories or include in your user info page.
Here is sample form for ljseek user.



Get your own code!

LjSEEK.COM - New Name, New Look, New Features.

Fri, 01 Jul 2005 04:12:25 GMT

Major service update took place. We have changed service name, improved
look and feel and added a lot of new features. More improvements are on
a way. The feature we've asked the most - ability to limit search to
your own journal is now here - take a look advanced search page.
We've also created ljseek - LiveJournal user, which will have news and
development progress announcements posted. Feel free to add it to your friends.

New Name

We've decided to change name from LjSearch.Net to LjSEEK.COM to avoid
confusion for our users - "ljsearch" is name used by some other
services.

New Look

We've changed the look and feel so it looks better and feels better.
If you spot any problems or inconveniences with new design please
us

New Features

- Limiting your search to certain users - You can limit your search to
certain users. You can provide list of users you want to limit your
search, you can also limit search to friends of certain users, for
example your own friends.

- Exact Phrase match: Search will only match exact phrases. This is
especially useful if you use our RSS search feed to avoid too many posts
getting where.

- Different Result Sort modes - depending on your search goals you may
get most relevant, newest, or oldest articles first. Combined sorting
is used by default to provide both new and relevant matches first.

- New dynamic excerpts

Now instead of showing you first few lines of the post we'll show the
piece of text where matched keywords are located.

LJSearch.Net is publicly available

Fri, 01 Jul 2005 04:10:49 GMT

Now we're searching some 26 million of posts by some 2 millions of LiveJournal users.
If you're not indexed yet let us know and we'll add your blog to the index. If you want to be removed, also just ask.
As you can see we're not advertisement free as LiveJournal but its inevitable - we do not have something like paid accounts
to earn money other way while full text searching is very resource intensive.

We're using MySQL Database and Sphinx Search ,
very fast Open Source Full Text search engine for MySQL. Without them it hardly would be possible.

If you have any thoughts about this service, any bug reports or wishes let us know. If you like this service please tell your friends
about it or link to us, more users will help to make it better.

Why one would want to search LiveJournal at all? Well this is very convenient way to find friends by checking what they write about,
you also may look up real user thoughts basically on any topic. Want to buy something? Try searching to find real user experiences,
rather than professional review. These are just couple of things we found it to be useful for. Bet you will find more.