Don't let search engines index your recipes
Please show respect for content owners. There is absolutely no valid reason for you to allow the hard work from other people to show up in search results under your domain. This is borderline reprehensible.
Please disallow crawling of the directory "/r/" in your robots.txt file.
Thanks for your note.
We have great respect for recipe content creators. As a search engine, it is our goal to send as much qualified traffic as we can to recipe websites. We’ll continue to introduce new features to optimize that effort, and we work very closely with recipe publishers to ensure they are happy with our treatment of their content.
As an example, we recently updated the print button on Foodily so that it clearly tells users they need to go to the original site to print, and it acts as a hot link to the source website.
Foodily only shows a small portion of the recipe instructions — including on pages with the /r/ url. In order to view the complete recipe instructions, users must visit the original recipe website. We will continue to evolve our experience by working with recipe publishers on how we can best support them.
Thanks again for your feedback. Please let us know if you have any other questions.
AdminChris Eldredge (Admin, Foodily) commented
Thanks for your note.
Foodily applies code which prevents other search engines from indexing recipes. You can see an example of this by viewing the source code on recipe details pages, such as http://www.foodily.com/r/zJKhjwDTO, which includes:
<meta name="robots" content="NOINDEX" />
In some cases, Foodily does have arrangements with publishers where this code does not appear.
I'm viewing your robots.txt and am still seeing this reprehensible behavior. What gives?
AdminPhillip (Admin, Foodily) commented
Hi Nathaniel. Coincidentally, we made the necessary changes last week. :) They'll show up when we push all our latest updates live (should be within the week if all goes well).
As we've said before, we're squarely in the camp of supporting content creators big and small and have never intended on growing search traffic off of other people's hard work.
It took us a while to deal with this but you should rest assured that we took your concerns very seriously, made sure we understood them fully, and made the changes as requested. And we hope that if you have other concerns you won't hesitate to share them with us.
Director of User Experience, Foodily
Do we have any update on this? I've spoken with a handful of prominent bloggers, and they were pretty floored by what is going on.
If this isn't resolved, it will be made a point of discussion at an upcoming food blog conference.
No worries Dan. Thanks for the upvotes!
Google strongly dislikes duplicate content, but detection isn't always perfect. The onus is on Foodily to behave properly. As it stands, Foodily is currently broadcasting to SE's that "Hey, this our content, please send people here to see it". So while they have a stated goal of helping content owners' sites (when people search directly on Foodily), they also surreptitiously work counter to that goal. All of these sites have their own web presences, and should be getting their traffic directly through search engines.
The kicker is that it takes -- I kid not -- a single line of code to fix.
This type of behavior, especially since it is clearly on purpose, really rubs me the wrong way.
Let's hope we get a substantive response.
I missed the point here, I understand what you are getting at now. And I have to say that I agree.
Its a risky strategy though - wouldn't google see that Foodily are serving duplicate content and remove the pages from its index anyway?
3 votes from me. And please accept my apologies for being so brash!
We are talking about different things. I am NOT talking about foodily indexing recipe pages, this is the point of the service, but rather Foodily allowing said pages to be indexed in SE's on Foodily's domain. There is a fundamental distinction. I understand your confusion -- it is confusing.
As per your analogy, it is apt, but flawed. Google indexes original content for their engine. This is wonderful. This is what Foodily does as well. All good. But Google does NOT then take said pages and serve them to other search engines to be indexed as Google content on the Google domain. You see the difference?
What planet are you on? Would you rather not have search engines? Have you written a similar notes to google, bing and yahoo? This is exactly the same as viewing cached versions of pages like you see on the mentioned search engines.
Leave the innovators alone, idiot.
Thanks for the response Chris.
I'm disappointed to learn this isn't merely an accident. I'm sure you respect content owners ... to a degree, but you are still endeavoring to get traffic from vectors other than people searching on foodily.com. This is *not* something search engines do, and even outside of engines, it is vehemently frowned upon.
You mention the fact you don't display full recipes -- that is great, but you didn't address the actual question of indexation.
Can you, or perhaps VP marketing/product, in plain terms state the justification for having other people's recipes show up in search results under the Foodily domain?
It is pretty common nowadays for companies that use such underhanded tactics to get called out publicly. Major SE's have recently specifically addressed issues of indexation of aggregated content. Foodily is new, but they will be made aware.
You guys have a great product, but employing such tactics *will* do more harm than good.
To clarify, the content owners being hurt by this are hundreds of supremely talented, hard working food bloggers and numerous test kitchen chefs from places like Bon Appétit and other publications.
I suspect (and hope) this is simply an oversight, and it needn't be brought to the attention of the food blogging community (and great sites like Epicurious).