FeedForAll Forum Index  
 Home  •  FAQ  •   Search   •  Register   •  Profile   •  Log in
Search found 15 matches
Author Message
Topic: Use RSS2HTML and RSSmesh locally without HTTP using CLI PHP
gnfti

Replies: 2
Views: 7247

PostForum: RSS2HTML (RSS to HTML Converter)   Posted: Mon Mar 22, 2010 9:35 pm   Subject: Use RSS2HTML and RSSmesh locally without HTTP using CLI PHP
Again, any help would be appreciated.
Topic: Use RSS2HTML and RSSmesh locally without HTTP using CLI PHP
gnfti

Replies: 2
Views: 7247

PostForum: RSS2HTML (RSS to HTML Converter)   Posted: Fri Mar 19, 2010 8:03 pm   Subject: Use RSS2HTML and RSSmesh locally without HTTP using CLI PHP
If anyone has any thoughts on this it would be greatly appreciated.
Topic: Use RSS2HTML and RSSmesh locally without HTTP using CLI PHP
gnfti

Replies: 2
Views: 7247

PostForum: RSS2HTML (RSS to HTML Converter)   Posted: Thu Mar 18, 2010 11:08 pm   Subject: Use RSS2HTML and RSSmesh locally without HTTP using CLI PHP
Hi all.

I would like to call RSS2HTML and RSSmesh locally on a web server (Linux, Apache, PHP5) without resorting to HTTP requests. All the XML files, RSSmesh config files and template files are available locally.

The rationale for this is that the scripts are used fairly intensively and I don't want the calls to count toward my bandwidth quota. Also, bypassing the web server can be faster in some instances.

I'm looking to run my Mesh scripts from the command line using Crontab like so:

/srv/customers/[$CUSTOMER#]/[$DOMAIN]/www/a/mesh-news-[$MESHNAME].php

They are correctly shebanged to use the server's PHP executable, CLI PHP is enabled, and other CLI PHP scripts are running happily this way from Crontab. I could just as well specify the executable with

/usr/local/bin/php -f /srv/customers/[$CUSTOMER#]/[$DOMAIN]/www/a/mesh-news-[$MESHNAME].php

, this has also been shown to work for other scripts.

Now, my question is how do I adapt the scripts so that they will play well with CLI PHP yet will not be passed over HTTP?

It is mainly the inclusion of the XML parser and other files that seem to throw me off. I understand how to pass arguments to PHP from the command line but I'm afraid I don't see how this works with recursion like the RSSmesh -> XML parser case or others.

Further notes:

-Permissions are correctly set on the output directories.
-I do not have direct access to the server's shell.
-If it matters, I'm also using RSS2HTML Pro and RSSfilter.

The above questions relates to the snag I ran into while trying to adapt RSSmesh for command line usage, but I will need to do the same for RSS2HTML next.

I suspect this can be overcome with some minor adaptations to the code, and I would appreciate some input from FFA as to how this can be realized.

Please let me know if you need any more information. Any help would be greatly appreciated.

Best,

Paul G. Hunt
Topic: help to show images
gnfti

Replies: 6
Views: 17646

PostForum: RSS2HTML (RSS to HTML Converter)   Posted: Sat Jul 19, 2008 9:52 am   Subject: help to show images
No problem. It seems like a good cause and an interesting site so I'm glad I could help.
Topic: help to show images
gnfti

Replies: 6
Views: 17646

PostForum: RSS2HTML (RSS to HTML Converter)   Posted: Sat Jul 19, 2008 8:01 am   Subject: help to show images
Ack, that didn't work.

[url=http://pipes.yahoo.com/pipes/pipe.info?_id=6sBoo5FV3RGKOhu9l7okhQ]Here's the pipe
[url=http://pipes.yahoo.com/pipes/pipe.run?_id=6sBoo5FV3RGKOhu9l7okhQ&_render=rss]Here's the output
Topic: help to show images
gnfti

Replies: 6
Views: 17646

PostForum: RSS2HTML (RSS to HTML Converter)   Posted: Sat Jul 19, 2008 7:58 am   Subject: help to show images
I built you a [url="http://pipes.yahoo.com/pipes/pipe.info?_id=6sBoo5FV3RGKOhu9l7okhQ"]Pipe.

[url="http://pipes.yahoo.com/pipes/pipe.run?_id=6sBoo5FV3RGKOhu9l7okhQ&_render=rss"]Here's the output as RSS.

I'd suggest you take a look at the source and see if you can figure out what I did. It's very useful to have a better knowledge of HTML and RSS/XML for this sort of thing.
Topic: help to show images
gnfti

Replies: 6
Views: 17646

PostForum: RSS2HTML (RSS to HTML Converter)   Posted: Fri Jul 18, 2008 9:42 am   Subject: help to show images
The images in the descriptions have relative paths, like

"resource/Ahmed_Naser[1].jpg"

What you need is absolute paths, like "http://www.waltainfo.com/resource/Ahmed_Naser[1].jpg".

It looks like you may be the owner of the feed; if so you should be able to fix this yourself. Otherwise, you could do some regex, but this requires some knowledge of PHP or what have you.
Topic: Feedburner files not read; Description elements not filtered
gnfti

Replies: 0
Views: 4776

PostForum: RSS2HTML (RSS to HTML Converter)   Posted: Wed Jul 16, 2008 11:09 am   Subject: Feedburner files not read; Description elements not filtered
A two-fer:

Previously, FFA described a known issue with (some?) Feedburner feeds not being parsed. In my setup, Feedburner content keeps disappearing and reappearing, and I can't find anything in the parser or feed that should cause this behaviour. Any ideas?

Secondly, rssFilter doesn't search ~~~ItemDescription~~~ using my filter script. An excerpt:

if ($currentItem->contains("~~~ItemTitle~~~", "obama", 0) ||
$currentItem->contains("~~~ItemDescription~~~", "obama", 0) ||
$currentItem->contains("~~~ItemContent~~~", "obama", 0) ||
$currentItem->contains("~~~ItemContentEncoded~~~", "obama", 0) ||
$currentItem->contains("~~~ItemDCSubject~~~", "obama", 0) ||

$currentItem->contains("~~~ItemTitle~~~", "bob barr", 0) ||
$currentItem->contains("~~~ItemDescription~~~", "bob barr", 0) ||
$currentItem->contains("~~~ItemContent~~~", "bob barr", 0) ||
$currentItem->contains("~~~ItemContentEncoded~~~", "bob barr", 0) ||
$currentItem->contains("~~~ItemDCSubject~~~", "bob barr", 0) ||

$currentItem->contains("~~~ItemTitle~~~", "mccain", 0) ||
$currentItem->contains("~~~ItemDescription~~~", "mccain", 0) ||
$currentItem->contains("~~~ItemContent~~~", "mccain", 0) ||
$currentItem->contains("~~~ItemContentEncoded~~~", "mccain", 0) ||
$currentItem->contains("~~~ItemDCSubject~~~", "mccain", 0) ||

$currentItem->contains("~~~ItemTitle~~~", "veep", 0) ||
$currentItem->c ...
Topic: One from pot A and one from pot B?
gnfti

Replies: 0
Views: 13130

PostForum: RSSFilter (Creates a filtered RSS feed)   Posted: Sun Jun 22, 2008 6:04 pm   Subject: One from pot A and one from pot B?
Is it possible to configure RSSFilter to select items containing at least one item from one set and at least one item from another set?

Example to illustrate what I'm trying to do:

Pot A
veepstakes
vp
running mate

Pot B
webb
pawlenty
bloomberg
jindal
fiorina
richardson
schweitzer
sebelius

-----

So, an item containing "vp" and "running mate" but none of the items in pot B would not match; an item containing, say, all of "fiorina", "veepstakes" and "bloomberg" would match. So would an item containing "running mate" and "sebelius".

Alternatively, is it possible to have one big pot of keywords and select for items that match at least two keywords from said pot?

Thanks in advance.
Topic: Conditional GET (HTTP 304) implementation.
gnfti

Replies: 2
Views: 9216

PostForum: RSS2HTML (RSS to HTML Converter)   Posted: Tue May 27, 2008 11:59 am   Subject: Conditional GET (HTTP 304) implementation.
This is why people should run their own web servers :)

Thanks for replying, Jim I don't really see the dynamically generated thing as a problem - it would still work on the ones that aren't dynamically generated, and every improvement is an improvement, bandwidth-wise.

So do you think it could be implemented?
Topic: Conditional GET (HTTP 304) implementation.
gnfti

Replies: 2
Views: 9216

PostForum: RSS2HTML (RSS to HTML Converter)   Posted: Thu May 22, 2008 11:09 am   Subject: Conditional GET (HTTP 304) implementation.
I believe this would serve to limit bandwidth use, especially when collecting data from a large number of sources, as I do.

I imagine this could be implemented at the parser level in a fashion not unlike this:

EXAMPLE RUN #1

1) Parser contacts server where source feed is located and requests "awesomefeed.xml"
2) Server says "HTTP 20x boo-ya"
3) Parser loads "awesomefeed.xml" and caches it locally as, say, "awesomefeed_20080522154101.xml" (YYYYMMDDhhmmss as an example, but it could be anything)
4) Parser parses feed and produces output for RSS2HTML, Mesh, Filter etc. to gnaw at
(5) Profit!)

EXAMPLE RUN #2

1) Parser contacts server where source feed is located and requests "awesomefeed.xml"
2) Server says "HTTP 20x boo-ya" (i.e. it was modified since last request)
3) Parser loads "awesomefeed.xml" and caches it locally as, say, "awesomefeed_20080522155101.xml".
4) Previously cached file(s) is/are deleted.
5) Parser parses feed and produces output
(6) More profit. Maybe start thinking about buying a yacht.)

EXAMPLE RUN #3

1) Parser contacts server where source feed is located and requests "awesomefeed.xml"
2) Server says "304 not modified"
3) Parser disconnects from remote server and loads local file "awesomefeed_20080522155101.xml"
4) Parser parses local feed file and produces output
(5) Profit margins declining from previous quarter, but you've successfully implemented conditional GET, and that's what matters.)

The parser would repeat run #3 (and thus read from the most recent cache file) for as long as it gets the 304 status code. Next time there's a 20x code, run #2 is executed, etc.

Because my site does not use caching in the FFA scripts but writes RSS2HTML output on the server (Ruby calls RSS2HTML using cron) and includes it in index.html, a variant without caching XML locally could be that upon encountering a 304, the script responsible will simply refuse to write output, leaving the previously written html file unchang ...
Topic: Some feeds not producing output, some are fine.
gnfti

Replies: 1
Views: 15314

PostForum: RSSmesh (Automatic Feed Aggregator)   Posted: Thu May 15, 2008 2:30 pm   Subject: Some feeds not producing output, some are fine.
Some feeds in my mesh scripts don't produce any output (when accessed directly they load fine). The config file is below, any ideas? I've commented out the feeds that *do* work for debugging purposes. The high item counts are for debugging as well, I'll change this back. Thanks in advance!

<?PHP

//
// Enable the caching for the resulting XML file.
$useOutputCaching = 0;

//
// The variable $outputCacheTTL controls how many seconds a cached copy of
// resulting RSS feed can be used before it must be recreated.
$outputCacheTTL = 1;

//
// $XMLTemplate is the XML file that contains the outer, , items
// for the produced feed. This MUST be a valid RSS 2.0 feed without any
// 's. It must also specify "UTF-8" encoding, since all the produced
// items will be in UTF-8.
$XMLTemplate = "rssMesh_blogs.xml";

//
// The variable $sortOutputByPubDate controls if the items in the resulting
// feed are sorted by their pubDate's, thus merging them together.
$sortOutputByPubDate = 1;

//
// The variable $feedList defines the feeds that will be used to create
// a new feed with.
$feedList = Array(

// Entry #1
//Array("fileURL" => "http://feeds.gawker.com/wonkette/excerpts.xml",
// "itemCount" => "3",
// "feedTTL" => "1",
// "feedExtra" => "Wonkette",
// "feedExtra1" => "http://www.wonkette.com/"
// )
//,

// Entry #2
Array("fileURL" => "http://feeds.feedburner.com/firedoglake/fdl",
"itemCount" => "33",
"feedTTL" => "1",
"feedExtra" => "Firedoglake",
"feedExtra1" => "http://firedoglake.com/"
)
,

// Entry #3
Array("fileURL" => "http://feeds.feedburner.com/TheCarpetbagg ...
Topic: Unix local path syntax to limit data traffic?
gnfti

Replies: 3
Views: 9615

PostForum: RSS2HTML (RSS to HTML Converter)   Posted: Sun Apr 20, 2008 4:31 pm   Subject: Unix local path syntax to limit data traffic?
Thanks for the reply. As I mentioned in my original post I had already tried that: turns out, the problem was the multiple iterations (i.e. mesh->filter->rss2html). You could run the RSS2HTML locally, but it still expects "http://..." for the filter and mesh scripts, and refuses to run otherwise.

I was unable to solve this with the given scripts. As a workaround, I moved all the PHP action to my home server and made it run and upload (FTP) via pycron and ruby.

It would be great if all the FFA scripts could be adapted to accept local Unix paths.

For cross-reference, [url=http://ask.metafilter.com/87977/My-websites-scripts-eat-up-too-much-bandwidth-How-do-I-resolve-this]a discussion of my situation on Ask Metafilter.
Topic: Unix local path syntax to limit data traffic?
gnfti

Replies: 3
Views: 9615

PostForum: RSS2HTML (RSS to HTML Converter)   Posted: Wed Apr 02, 2008 11:56 am   Subject: Unix local path syntax to limit data traffic?
Hi all,

I run Electicker, which uses a truckload of FFA scripts to gather data. To limit page load times I cache everything using a Ruby script that runs every few minutes, like so:

data = open("http://www.electicker2008.com/php/rss2html2.php?XMLFILE=http://www.electicker2008.com/php/filter.php?feed=http://www.electicker2008.com/php/mesh-headlines.php&TEMPLATE=http://www.electicker2008.com/php/news-relative-template.html&NOFUTUREITEMS=1&MAXITEMS=8&ItemDescriptionLength=100").read
File.open(this_directory + "/headlines.html", 'w') do |f|
f.write data
end

You will notice the multiple steps it calls to render the data, i.e. Mesh -> Filter -> RSS2HTML.

Now, because the URLs here are all in "http://" format, it generates a huge amount of data, which counts towards (and well over) my traffic quota. (Electicker runs more than 20 versions of the above code.)

My web host wants me to change the URLs to local paths, i.e.

/srv/customers/[$MYCUSTNO]/electicker2008.com/www/php/[$FILE]

How do I format this in the Ruby script so it works? I'm getting 500 errors right now. Thanks in advance for any help.
Topic: timeago / timesince / fuzzy time / natural language time
gnfti

Replies: 1
Views: 5567

PostForum: RSS2HTML (RSS to HTML Converter)   Posted: Tue Jan 15, 2008 11:01 pm   Subject: timeago / timesince / fuzzy time / natural language time
I'd like to change timestamps into Timeago, Timesince, or relative time - there seem to be many different names - i.e. "12 minutes ago" or "3 hours ago". I've found a few code examples which seem elegant solutions, such as [url=http://nl.php.net/manual/en/function.time.php#74713]this and [url=http://www.nicklarson.com/2007/09/26/calculating-the-freshness-of-a-time-stamp/]this one.

Now, how do I integrate this into the RSS2HTML code?
 

 Jump to:   


Powered by phpBB © 2001, 2002 phpBB Group :: FI Theme