Skip to content

Commit

Permalink
Version 1.4
Browse files Browse the repository at this point in the history
- NOTICE: config.default.php has changed, re-create your config.php!!
- [fix] Footer no longer overlaps results
- [fix] Search navigation no longer bunched up on smaller displays
- [fix] Double search type when searching from start page
- [new] Filter for additional/different headers per cURL request
- [new] Image search via Openverse API (Access token and cronjob required, see installation instructions)
- [new] Image search via Qwant API
- [new] Web (recent news) search via Qwant API
- [tweak] Merged 'cache' option into 'cache-type', see config.default.php for details
- [tweak] Better filtering for duplicate web results
- [tweak] File size formatting for images more uniform
- [tweak] Optimized curl_multi_exec handling
- [tweak] Improved SEO headers
- [tweak] Layout tweaks and optimizations for search results, header and footer
- [tweak] Removed redundant HTML, CSS and some PHP
- [tweak] MagnetDL search disabled by default because of Cloudflare (Will probably be removed in future version)
- [tweak] Removed non-functional magnet trackers
- [tweak] Added 15 extra public magnet trackers
- [change] Removed Ecosia support
- [change] Removed Reddit support
- [change] Removed 1337x support
- [change] Removed MagnetDL support
  • Loading branch information
adegans committed May 17, 2024
1 parent aab57e1 commit 32c2792
Show file tree
Hide file tree
Showing 37 changed files with 1,400 additions and 1,053 deletions.
299 changes: 148 additions & 151 deletions assets/css/styles.css

Large diffs are not rendered by default.

Binary file added assets/images/goosle.webp
Binary file not shown.
191 changes: 81 additions & 110 deletions config.default.php
Original file line number Diff line number Diff line change
Expand Up @@ -11,154 +11,106 @@
------------------------------------------------------------------------------------ */

/* ------------------------------------------------------------------------------------
SITEURL:
Set the base domain name for your Goosle setup (ex. example.com, something.example.com, example.com/something/) so that internal links will work correctly.
HASH:
A simple lowercase passphrase (something simple like: j9fg-i2du-er6m or 1846) used for caching results. This helps to differentiate between instances on the same server.
You can also add it to your url/bookmark as a simple passphrase to keep unwanted users out.
A simple lowercase passphrase, something simple like: j9fg-i2du-er6m or 1846.
Used for caching results and optionally for accessing Goosle (See HASH_AUTH option).
HASH AUTH:
HASH_AUTH:
Use the above hash as a simple passphrase.
Using it as a passphrase lets you host Goosle on a public server without providing a public service.
Using it as a passphrase lets you host Goosle on a public facing server without providing a public service.
This is useful for if just you and some friends or family should be able to use Goosle from anywhere.
"off" Don't use the hash as a password.
"on" Use the hash as a password.
Usage: https://example.com/?a=1234567890
Disclaimer: This is not meant to 'hack proof' or truly secure the setup. Just a simple token to keep surface level prying eyes out.
CACHE:
It is highly recommended to enable caching as it'll speed up repeat searches by a lot.
Disclaimer: This is not meant to 'hack proof' or truly secure the setup. Just a simple token to keep surface level prying eyes out.
CACHE_TYPE:
Choose how to cache results. The cache is NOT unique per user but shared between all users. Different users searching for the exact same thing get the same results.
Default caching method is APCu. Alternatively, you can store the results in text files in the /cache/ folder.
Ignored if above 'cache' option is set to off.
"apcu" (Recommended) faster, utilize memory.
"file" Store results in text files.
It is highly recommended to enable caching as it will speed up repeat searches by a lot.
The cache is NOT unique per user but shared between all users. Different users searching for the exact same thing get the same results.
Caching can be done in memory with APCu or as temporary files in the /cache/ folder.
"off" No caching.
"file" Store results in text files (Default).
"apcu" Faster, utilizes memory.
CACHE_TIME:
Minutes the result should be cached. Accepts a numeric value between 1 and 720.
APCu stores in memory, using a longer time takes up more of it. It is recommended to not exceed 30 minutes for it.
APCu stores in memory, using a longer time takes up more of it. It is recommended to not exceed 30 minutes for APCu.
The file cache is only limited by your hosting storage space and can safely be much much longer if you want.
To not show outdated results the 'limit' is 720 minutes, which equals 12 hours.
Ignored if above 'cache' option is set to off.
ENABLE IMAGE SEARCH:
Enable or disable image searches - Search results are provided by Yahoo! Images.
ENABLE MAGNET SEARCH:
Enable or disable searching for magnet links on torrent websites.
ENABLE SEARCH ENGINES:
Enable or disable search engines.
ENABLE MAGNET CRAWLERS:
Enable or disable crawlers to pull magnet links from.
Ignored if above 'CACHE_TYPE' option is set to off.
/* ------------------------------------------------------------------------------------
LANGUAGE:
DuckDuckGo, Google and Ecosia are language agnostic. But they DO profile you for your locale.
For example: Me searching with english terms has me seeing Spanish results because I live in Mexico. This setting should minimize that for supported engines.
DuckDuckGo uses language regions as opposed to a simpler language choice. See if your region is available - https://duckduckgo.com/duckduckgo-help-pages/settings/params/.
Google's language option breaks reasonable results and other options like verbatim mode and some other settings. So you'll have to rely on Google picking up on the query language.
DuckDuckGo and Google are mostly language agnostic.
To not fit the USA mold, Goosle defaults to the United Kingdom for english results.
Google has no language setting because as soon as you specify it all 'anonymous' settings stop working.
DuckDuckGo uses language regions and defaults to the United Kingdom. To change it see if your region is available - https://duckduckgo.com/duckduckgo-help-pages/settings/params/.
Wikipedia needs to be told which language you want. This changes the search url. Use any of their supported languages (en, es, fr, nl, etc.)
Qwant uses a locale similar to DuckDuckGo. Available locales are: bg_bg, br_fr, ca_ad, ca_es, ca_fr, co_fr, cs_cz, cy_gb, da_dk, de_at, de_ch, de_de, ec_ca, el_gr, en_au, en_ca, en_gb, en_ie, en_my, en_nz, en_us, es_ad, es_ar, es_cl, es_co, es_es, es_mx, es_pe, et_ee, eu_es, eu_fr, fc_ca, fi_fi, fr_ad, fr_be, fr_ca, fr_ch, fr_fr, gd_gb, he_il, hu_hu, it_ch, it_it, ko_kr, nb_no, nl_be, nl_nl, pl_pl, pt_ad, pt_pt, ro_ro, sv_se, th_th, zh_cn, zh_hk.
SOCIAL MEDIA RELEVANCE:
Show social media results lower in the combined results if you don't value such results.
Downranked results include: Facebook, Instagram, Twitter, Snapchat, TikTok, LinkedIn and Reddit.
!!CAREFUL!! This is a blanket setting, if what you're searching for primarily has social media links then less relevant results may show first.
Accepts a numeric value between 1 and 10. With 10 having *NO* effect on the rank, and 0 not ranking the link at all (shows very very low in the results)
SHOW SEARCH SOURCE:
Show which search engine(s) came up with the result.
SHOW SEARCH RANK:
When search source is enabled, show the rank Goosle gave the result.
IMDB ID SEARCH:
Highlight imdb results if it's a tv-show or movie.
Handy for finding better results for specific tv-shows through EZTV and The Pirate Bay.
PASSWORD GENERATOR
Show a password generator on the Goosle home page.
SPECIAL:
Enable or disable special searches that show up before search results.
SHOW ZERO SEEDERS:
Set to "on" to include results with 0 seeders (slow or stale downloads). Off to exclude these results.
YTS HIGHLIGHT:
If you've enabled the YTS special search, you can also choose what it should show. The 8 most [insert choice] movies.
"date_added" = Newest movies (Default).
"rating" = Highest rated movies as per imdb.
"download_count" = Most downloaded movies.
BLOCK 1337x CATEGORIES:
Add category IDs of 1337x categories, check /engines/magnet/1337x.php for a list of known categories.
Accepts a basic numeric array, comma separated.
BLOCK PIRATEBAY CATEGORIES:
Add category IDs of Pirate Bay categories, check /engines/magnet/thepiratebay.php for a list of known categories.
Accepts a basic numeric array, comma separated.
BLOCK YTS CATEGORIES:
Add category names as keywords, eg; "thriller", "war".
Movies can be in multiple categories, if a movie is in 5 categories it only has to match one to be filtered out.
Accepts a basic array of keywords, comma separated.
Downranked results include websites like Facebook, Instagram, Twitter, Snapchat, TikTok, LinkedIn and Reddit.
!! CAREFUL !! This is a blanket setting, if what (or who) you're searching for primarily has social media links then less relevant results may show first.
Accepts a numeric value between 1 and 10. With 10 having *NO* effect on the rank, and 0 not ranking the link at all (shows very very low in the results).
/* ------------------------------------------------------------------------------------
USER AGENTS:
Add more or less user agents to the list. Keep at least one!
Add more or less user agents to the list but keep at least one!
On every search Goosle picks one at random to identify as.
Keep them generic to prevent profiling, but also so that the request comes off as a generic boring browser and not as a server/crawler.
Safari, Firefox and Internet Explorer/Edge should be safe to use.
Safari, Firefox and Internet Explorer (Yes that's old!) should be safe to use.
Chrome may attract attention because of the lack of Chrome information (tracking) aside from the user agent. The search engine may know something is 'weird'.
Opera/Edge/Brave and many others use Chrome under the hood and are not a good pick for that reason.
Mobile agents may work, but some services like Wikipedia are a bit picky when it comes to answering API calls. Mobile users generally do not use APIs, so they may block your search.
Mobile user agents may work, but some services like Wikipedia are a bit picky when it comes to answering API calls.
Mobile users generally do not use APIs, so they may block your search or show a trimmed version of results.
MAGNET TRACKERS:
Only used for The Pirate Bay, LimeTorrents and YTS.
These are added to the magnet links Goosle creates by itself.
Generally you do not need to change these.
These are added to the magnet links Goosle creates. You can add more or replace the existing ones if you know what you're doing.
Accepts a basic array of strings (tracker urls), comma separated.
Currently only The Pirate Bay, LimeTorrents and YTS use generated magnet links.
You can add more or replace the existing ones if you know what you're doing. But keep at least one, preferably 3-5+.
------------------------------------------------------------------------------------ */

return (object) array(
"hash" => "j9fg-i2du-er6m",
// ALL OPTIONS ARE REQUIRED, EVEN IF YOU DO NOT USE THE FEATURE. EMPTY VALUES OR MISSING SETTINGS CAUSE ISSUES!!!
"siteurl" => "example.com", // Make sure this is accurate
"hash" => "j9fg-i2du-er6m", // Some kind of alphanumeric password-like string, used for caching and optionally for access to Goosle
"hash_auth" => "off", // Default: off
"cache" => "off", // Default: off
"cache_type" => "apcu", // Default: apcu
"cache_type" => "file", // Default: file
"cache_time" => 30, // Default: 30 (Minutes)

"enable_image_search" => "on", // Default: on
"enable_magnet_search" => "on", // Default: on
"enable_duckduckgo" => "on", // Default: on
"enable_google" => "on", // Default: on
"enable_reddit" => "on", // Default: on
"enable_qwantnews" => "on", // Default: on
"enable_wikipedia" => "on", // Default: on
"enable_ecosia" => "off", // Default: on
// Site uses some kind of bot detector preventing crawler from working reliably since Feb 1, 2024, remove support in future release?)

"enable_image_search" => "on", // Default: on (Disables all image search regardless of settings for individual engines)
"enable_yahooimages" => "on", // Default: on
"enable_openverse" => "off", // Default: off (Requires API token, see readme for details)
"enable_qwant" => "on", // Default: on

"enable_magnet_search" => "on", // Default: on (Disables all image search regardless of settings for individual engines)
"enable_eztv" => "on", // Default: on
"enable_limetorrents" => "on", // Default: on
"enable_nyaa" => "on", // Default: on
"enable_piratebay" => "on", // Default: on
"enable_yts" => "on", // Default: on
"enable_magnetdl" => "on", // Default: on
"enable_nyaa" => "on", // Default: on
"enable_eztv" => "on", // Default: on
"enable_l33tx" => "off", // Default: off
// Site now uses cloudflare preventing crawler from working since Jan 20, 2024, remove support in future release?)

"duckduckgo_language" => "uk-en", // Default: uk-en (United Kingdom)
"wikipedia_language" => "en", // Default: en (English)
"qwant_language" => "en_gb", // Default: en_gb (United Kingdom)

"social_media_relevance" => 8, // Default: 8
"show_reddit_nsfw" => "on", // Default: on
"show_search_source" => "on", // Default: on
"show_search_rank" => "off", // Default: off
"imdb_id_search" => "off", // Default: off
Expand All @@ -174,11 +126,15 @@

"show_zero_seeders" => "on", // Default: on
"yts_highlight" => "date_added", // Default: "date_added"
"leetx_categories_blocked" => array(3, 7, 47), // Default: 3, 7, 47
"piratebay_categories_blocked" => array(206, 210), // Default: 206, 210
"yts_categories_blocked" => array("horror"), // Default: "horror"

"user_agents" => array(
"Lynx/2.8.5rel.1 libwww-FM/2.14 SSL-MM/1.4.1", // Linux, Lynx browser 2.8.5
"Lynx/2.8.9rel.1 libwww-FM/2.14 SSL-MM/1.4.1", // Linux, Lynx browser 2.8.9
"Lynx/2.8.6rel.4 libwww-FM/2.14 SSL-MM/1.4.1", // Linux, Lynx browser 2.8.6
"TinyBrowser/2.0 (TinyBrowser Comment; rv:1.9.1a2pre) Gecko/20201231", // Linux, Tinybrowser 2
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15) DuckDuckGo/7 Safari/605.1.15", // macOS 10.15, DuckDuckGo 7
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15) Gecko/20100101 Firefox/119.0", // macOS 10.15, Firefox 119
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) Gecko/20100101 Firefox/116.0", // Windows 10, Firefox 116
"Mozilla/5.0 (X11; Ubuntu; Linux x86_64) Gecko/20100101 Firefox/83.0", // Linux Ubuntu, Firefox 83
Expand All @@ -187,11 +143,26 @@

"magnet_trackers" => array(
"http://nyaa.tracker.wf:7777/announce",
"http://tracker.openbittorrent.com:80/announce",
"udp://tracker.opentrackr.org:1337/announce",
"udp://exodus.desync.com:6969/announce",
"udp://tracker.opentrackr.org:1337/announce",
"udp://exodus.desync.com:6969/announce",
"udp://tracker.torrent.eu.org:451/announce",
"udp://opentracker.i2p.rocks:6969/announce",
"udp://open.demonii.com:1337/announce",
"udp://open.stealth.si:80/announce",
"udp://tracker.moeking.me:6969/announce",
"udp://explodie.org:6969/announce",
"udp://tracker1.bt.moack.co.kr:80/announce",
"udp://tracker.theoks.net:6969/announce",
"udp://tracker-udp.gbitt.info:80/announce",
"https://tracker.tamersunion.org:443/announce",
"https://tracker.gbitt.info:443/announce",
"udp://tracker.tiny-vps.com:6969/announce",
"udp://tracker.dump.cl:6969/announce",
"udp://tamas3.ynh.fr:6969/announce",
"udp://retracker01-msk-virt.corbina.net:80/announce",
"udp://open.free-tracker.ga:6969/announce",
"udp://epider.me:6969/announce",
"udp://bt2.archive.org:6969/announce",
)
);
?>
?>
111 changes: 111 additions & 0 deletions engines/image/openverse.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
<?php
/* ------------------------------------------------------------------------------------
* Goosle - A meta search engine for private and fast internet fun.
*
* COPYRIGHT NOTICE
* Copyright 2023-2024 Arnan de Gans. All Rights Reserved.
*
* COPYRIGHT NOTICES AND ALL THE COMMENTS SHOULD REMAIN INTACT.
* By using this code you agree to indemnify Arnan de Gans from any
* liability that might arise from its use.
------------------------------------------------------------------------------------ */
class OpenverseRequest extends EngineRequest {
public function get_request_url() {

$query_terms = substr($this->query, 0, 200);

// Safe search override
$safe = "0"; // No mature results
if(strpos($query_terms[0], "safe") !== false) {
$switch = explode(":", $query_terms[0]);

if(!is_numeric($switch[1])) {
$safe = (strtolower($switch[1]) == "off") ? "1" : "0";
$this->query = implode(" ", array_slice($query_terms, 1));
}
}

// q = query
// format = json
// mature = 1 / 0
// page_size = 80 (int)

$args = array("q" => $query_terms, "format" => "json", "mature" => $safe, "page_size" => 50);
$url = "https://api.openverse.org/v1/images/?".http_build_query($args);

unset($query_terms, $switch, $safe, $max_results, $args);

return $url;
}

public function get_request_headers() {
$token_file = ABSPATH.'cache/token.data';
$token = unserialize(file_get_contents($token_file));

return array(
'Accept' => 'application/json, */*;q=0.8',
'Content-type' => 'application/x-www-form-urlencoded',
'Authorization' => 'Bearer '.$token['openverse']['access_token'],
'Accept-Language' => null,
'Accept-Encoding' => null,
'Connection' => null,
'Sec-Fetch-Dest' => null,
'Sec-Fetch-Mode' => null,
'Sec-Fetch-Site' => null
);
}

public function parse_results($response) {
$results = array();
$json_response = json_decode($response, true);

// No response
if(empty($json_response)) return $results;

// Set base rank and result amound
$rank = $results['amount'] = count($json_response['results']);

// Nothing found
if($results['amount'] == 0) return $results;

// Use API result
foreach ($json_response['results'] as $result) {
// Deal with optional or missing data
$dimensions_w = (!empty($result['width'])) ? sanitize($result['width']) : "";
$dimensions_h = (!empty($result['height'])) ? sanitize($result['height']) : "";
$filesize = (!empty($result['filesize'])) ? sanitize($result['filesize']) : "";
$link = (!empty($result['url'])) ? sanitize($result['url']) : "";

$image_full = (!empty($result['foreign_landing_url'])) ? sanitize($result['foreign_landing_url']) : "";
$image_thumb = (!empty($result['thumbnail'])) ? sanitize($result['thumbnail']) : $image_full;
$alt = (!empty($result['title'])) ? sanitize($result['title']) : "";

// Add attribution to alt text?
$creator = (!empty($result['creator'])) ? " by ".sanitize($result['creator']) : "";
$alt = (!empty($creator)) ? $alt.$creator : $alt;

// Process result
$filesize = intval(preg_replace('/[^0-9]/', '', $filesize));

// filter duplicate IMAGE urls/results
if(!empty($results['search'])) {
if(in_array($image_full, array_column($results['search'], "image_full"))) continue;
}

$results['search'][] = array ("id" => uniqid(rand(0, 9999)), "source" => "Openverse", "image_thumb" => $image_thumb, "alt" => $alt, "image_full" => $image_full, "width" => $dimensions_w, "height" => $dimensions_h, "filesize" => $filesize, "webpage_url" => $link, "engine_rank" => $rank);
$rank -= 1;
unset($url_data, $usable_data, $dimensions_w, $dimensions_h, $filesize, $link, $image_full, $alt, $image_thumb);
}
unset($json_response, $rank);

// Add error if there are no search results
if(empty($results['search'])) {
$results['error'] = array(
"message" => "No results found. Please try with less or different keywords!"
);
}

return $results;
}
}
?>
Loading

0 comments on commit 32c2792

Please sign in to comment.