10. February 2018

Screen Scraping with Goutte

We all have been in situations were we need content or information from a connected website, but have no access to a REST Api or any other backend feed.

In these cases screen scraping is the only option to get needed information to finalize an integration. You can do that directly in CURL, but that can be tedious. Far easier to use a nicely packaged solution that combines a component that simulates web browser behavior and a component that eases DOM navigation for HTML and XML documents. Meet Goutte!

STEP 0

Install via composer.

composer require fabpot/goutte

1	composer require fabpot/goutte

STEP 1

$client = new Client();
$crawler = $client->request('GET', 'https://www.page/login.php');

// select the form and fill in some values
$form = $crawler->selectButton('Login')->form();
$form['f_loginname'] = 'HelloMe';
$form['f_loginpass'] = 'securepass';

// submit that form
$crawler = $client->submit($form);

// go to next page    
$crawler = $client->request('GET', 'https://www.page.de/overview.php');

$client = new Client();

$crawler = $client->request('GET', 'https://www.page/login.php');

// select the form and fill in some values

$form = $crawler->selectButton('Login')->form();

$form['f_loginname'] = 'HelloMe';

$form['f_loginpass'] = 'securepass';

// submit that form

$crawler = $client->submit($form);

// go to next page

$crawler = $client->request('GET', 'https://www.page.de/overview.php');

STEP 2

Get the data you need.

// loop over html and filter out what you need

$crawler->filter('table.clients tr')->each(function ($node) {

  $node->filter('td')->each(function ($sub_node) {
    echo $sub_node->html();
  }
}

// loop over html and filter out what you need

$crawler->filter('table.clients tr')->each(function ($node) {

$node->filter('td')->each(function ($sub_node) {

echo $sub_node->html();

}

Goutte @ Github
BrowserKit Documentation
DOM Crawler Documentation

Enjoy coding …

27. January 2018

Audiomanager for text-to-speech cloud services

I have been working on a project , that relies on text-to-speech cloud services for a lot of things.

Audiomanager abstracts access to popular text-to-speech cloud services like Google, Ivona, Amazon Polly… Really a timesaver, when evaluating different services.

Polly Example

$adapter = new \AudioManager\Adapter\Polly();
$adapter->getOptions()->initialize()
    ->setVersion('latest')
    ->setRegion('us-west-2')
    ->setCredentials()
        ->setKey('...')
        ->setSecret('...');

$adapter->getOptions()->setOutputFormat('...'); //Default 'mp3'
$adapter->getOptions()->setLexiconNames('...');
$adapter->getOptions()->setSampleRate('...'); //Default '16000'
$adapter->getOptions()->setTextType('...'); //Default 'text'
$adapter->getOptions()->setVoiceId('...'); //Default 'Salli'

$adapter = new \AudioManager\Adapter\Polly();

$adapter->getOptions()->initialize()

->setVersion('latest')

->setRegion('us-west-2')

->setCredentials()

->setKey('...')

->setSecret('...');

$adapter->getOptions()->setOutputFormat('...'); //Default 'mp3'

$adapter->getOptions()->setLexiconNames('...');

$adapter->getOptions()->setSampleRate('...'); //Default '16000'

$adapter->getOptions()->setTextType('...'); //Default 'text'

$adapter->getOptions()->setVoiceId('...'); //Default 'Salli'

GitHub

Enjoy coding ….

27. January 2018

Never store sensitive credentials in your code again – PHP dotenv

PHP dotenv loads environment variables from .env to getenv(), $_ENV and $_SERVER automagically.

You should never store sensitive credentials in your code. Anything that is likely to change between deployment environments – such as database credentials or credentials for 3rd party services – should be extracted from the code into environment variables.

Add your application configuration to a .env file in the root of your project. Make sure the .env file is added to your .gitignore so it is not being checked-in.

DEFINE

S3_BUCKET="dotenv"
SECRET_KEY="souper_seekret_key"

1 2	S3_BUCKET="dotenv" SECRET_KEY="souper_seekret_key"

LOAD

$dotenv = new Dotenv\Dotenv(__DIR__);
$dotenv->load();

// OR WITH filename

$dotenv = new Dotenv\Dotenv(__DIR__, 'myconfig');
$dotenv->load();

$dotenv = new Dotenv\Dotenv(__DIR__);

$dotenv->load();

// OR WITH filename

$dotenv = new Dotenv\Dotenv(__DIR__, 'myconfig');

$dotenv->load();

ACCESS

$s3_bucket = getenv('S3_BUCKET');
$s3_bucket = $_ENV['S3_BUCKET'];
$s3_bucket = $_SERVER['S3_BUCKET'];

$s3_bucket = getenv('S3_BUCKET');

$s3_bucket = $_ENV['S3_BUCKET'];

$s3_bucket = $_SERVER['S3_BUCKET'];

GitHub

Enjoy coding and be secure …

27. January 2018

Generate Unique User Nicenames for WordPress

Simple little snippet, that can easily be used in conjunction with wp_update_user()

/* Generate Unique User Nicename */
function generate_unique_user_nicename( $x ) {
	// set up args to query
	$args = array(
		'search'         => sanitize_title( $x ),
		'search_columns' => array( 'user_nicename' )
	);
	// query for user
	$user_url_exists = new WP_User_Query( $args );
	$results = $user_url_exists->get_results();
	// if user url is found, check for new one
	if( !empty( $results ) ) {
		$i = substr( sanitize_title( $x ), intval( strrpos( sanitize_title( $x ) ,'-' ) ) + 1 ); 
		if( is_numeric( $i ) ) {
			$i++;
			$y = substr($x, 0, strrpos($x,'-' ));
			$company_name = $y . '-' . $i;
		} else {
			$company_name = $x . '-1';
		}
		$new_company_name = sanitize_title( $company_name );
		return generate_unique_user_nicename( $new_company_name );
	} else {
		return sanitize_title( $x );
	}
}

/* Generate Unique User Nicename */

function generate_unique_user_nicename( $x ) {

// set up args to query

$args = array(

'search' => sanitize_title( $x ),

'search_columns' => array( 'user_nicename' )

);

// query for user

$user_url_exists = new WP_User_Query( $args );

$results = $user_url_exists->get_results();

// if user url is found, check for new one

if( !empty( $results ) ) {

$i = substr( sanitize_title( $x ), intval( strrpos( sanitize_title( $x ) ,'-' ) ) + 1 );

if( is_numeric( $i ) ) {

$i++;

$y = substr($x, 0, strrpos($x,'-' ));

$company_name = $y . '-' . $i;

} else {

$company_name = $x . '-1';

}

$new_company_name = sanitize_title( $company_name );

return generate_unique_user_nicename( $new_company_name );

} else {

return sanitize_title( $x );

}

Enjoy coding …

23. January 2018

Cleaning up the WordPress Feed (Validation)

When using your main content feed to share posts via buffer or other services, it is crucial that your feed validates cleanly.

There are always things in the generated WordPress content that can break your feed validation. Here some things to cleanup or alter your delivered feed content.

CONDITIONAL TAGS

is_feed()
Check for syndication request. This tag is not typically used by users; it is used internally by WordPress and is available for Plugin Developers.

Check for feed syndication in your themes functions.php

if (is_feed()) {
    /.. tweaks go here ../
}

if (is_feed()) {

/.. tweaks go here ../

}

REMOVE IFRAMES

function rss_noiframe($content) {
        
    $content = preg_replace( '/<iframe(.*)\/iframe>/is', '', $content );

    return $content;
}

add_filter('the_excerpt_rss', 'rss_noiframe');
add_filter('the_content_feed', 'rss_noiframe');

function rss_noiframe($content) {

$content = preg_replace( '/<iframe(.*)\/iframe>/is', '', $content );

return $content;

}

add_filter('the_excerpt_rss', 'rss_noiframe');

add_filter('the_content_feed', 'rss_noiframe');

REMOVE / DISABLE COMMENTS

function rss_nocomments($content) {
    
     global $post;
      $post->comment_status="closed";
   
  
}

add_filter('the_excerpt_rss', 'rss_nocomments');
add_filter('the_content_feed', 'rss_nocomments');

function rss_nocomments($content) {

global $post;

$post->comment_status="closed";

}

add_filter('the_excerpt_rss', 'rss_nocomments');

add_filter('the_content_feed', 'rss_nocomments');

REMOVE RESPONSIVE IMAGES

The sizes attribute breaks feed validation, here how to clean it up.

function no_responsive_image_feeds() {
    add_filter( 'max_srcset_image_width', function() {
       return 1;
    } );
}

add_action('rss2_head', 'no_responsive_image_feeds' );
add_action('atom_head', 'no_responsive_image_feeds' );
add_action('rss_head', 'no_responsive_image_feeds' );

function no_responsive_image_feeds() {

add_filter( 'max_srcset_image_width', function() {

return 1;

} );

}

add_action('rss2_head', 'no_responsive_image_feeds' );

add_action('atom_head', 'no_responsive_image_feeds' );

add_action('rss_head', 'no_responsive_image_feeds' );

INCLUDE PAGES IN POST FEED

if (is_feed()) {
function feedFilter($query) {
	if ($query->is_feed) {
		$query->set('post_type','any');
		}
	return $query;
}
add_filter('pre_get_posts','feedFilter');
}

if (is_feed()) {

function feedFilter($query) {

if ($query->is_feed) {

$query->set('post_type','any');

}

return $query;

}

add_filter('pre_get_posts','feedFilter');

}

ADJUST AMOUNT OF POSTS

if (is_feed()) {

function feedFilter($query) {
	if ($query->is_feed) {
		$query->set('posts_per_page','11');
	}
 
	return $query;
}
add_filter('pre_get_posts','feedFilter');

}

if (is_feed()) {

function feedFilter($query) {

if ($query->is_feed) {

$query->set('posts_per_page','11');

}

return $query;

}

add_filter('pre_get_posts','feedFilter');

}

LIMIT BY CATEGORY

if (is_feed()) {
function feedFilter($query) {
	if ($query->is_feed) {
		$query->set('category_name', 'my-special-cat');
	}
 
	return $query;
}
add_filter('pre_get_posts','feedFilter');
}

if (is_feed()) {

function feedFilter($query) {

if ($query->is_feed) {

$query->set('category_name', 'my-special-cat');

}

return $query;

}

add_filter('pre_get_posts','feedFilter');

}

Enjoy coding …

30. August 2017

GreenApe is breathing again

After some downtime, GreenApe is breathing again. I revived the brand with a good friend of mine and we will be reopening shop options shortly.

Michael and I have been friends for a long time. We have been working on many different projects over the years.

He launched GreenApe in 2011 and I helped him with his first steps. A couple of months ago we decided to merge our competences and expand what GreenApe offers and stands for.

From the website: “The GreenApe brand was established in 2011. GreenApe’s career began with the 1st Single Malt Whisky Coffee.

As the first of its kind, our coffee is refined with Original Single Malt Whisky. To this day, he pampers many connoisseurs and gourmets with his unique taste. Now there is another reason to rejoice.

From now on, we are continuously expanding the GreenApe product world with several stylish gadgets and useful accessories. For you this means that you will be able to discover even more beautiful, special or practical things in the future.“

GreenApe is all about lifestyle & leisure products, fun gadgets and unique food & drinks.

18. June 2017

Unicode slugs with ease

What is a slug ?
A slug is a part of a URL which identifies a particular page on a website in a form readable by users. It’s the nice part of the URL, which explains the page’s content. In WordPress, it’s the editable part of your URL that you can edit when writing a new post.

nSlug allows to create those urls for any language easily.

use NSlug\SlugGenerator;

$options = [
    'limit'     => null,
    'delimiter' => '-',
    'lowercase' => true,
];

$slugGenerator = new SlugGenerator($options);

// English strings.
echo $slugGenerator->generate('Hello World'); // Prints 'hello-world'.
echo $slugGenerator->generate('Life is too short!'); // Prints 'life-is-too-short'.

// Arabic strings.
echo $slugGenerator->generate('صباح جميل'); // Prints 'صباح-جميل'.
echo $slugGenerator->generate('مساء الخير'); // Prints 'مساء-الخير'.

use NSlug\SlugGenerator;

$options = [

'limit' => null,

'delimiter' => '-',

'lowercase' => true,

];

$slugGenerator = new SlugGenerator($options);

// English strings.

echo $slugGenerator->generate('Hello World'); // Prints 'hello-world'.

echo $slugGenerator->generate('Life is too short!'); // Prints 'life-is-too-short'.

// Arabic strings.

echo $slugGenerator->generate('صباح جميل'); // Prints 'صباح-جميل'.

echo $slugGenerator->generate('مساء الخير'); // Prints 'مساء-الخير'.

GitHub

14. May 2017

Fixing broken HTTP Basic Authentication (CGI/FastCGI under Apache)

PHP

if( !isset($_SERVER['PHP_AUTH_USER']) )
{
    if (isset($_SERVER['HTTP_AUTHORIZATION']) && (strlen($_SERVER['HTTP_AUTHORIZATION']) > 0)){
        list($_SERVER['PHP_AUTH_USER'], $_SERVER['PHP_AUTH_PW']) = explode(':', base64_decode(substr($_SERVER['HTTP_AUTHORIZATION'], 6)));
        if( strlen($_SERVER['PHP_AUTH_USER']) == 0 || strlen($_SERVER['PHP_AUTH_PW']) == 0 )
        {
            unset($_SERVER['PHP_AUTH_USER']);
            unset($_SERVER['PHP_AUTH_PW']);
        }
    }
}

if( !isset($_SERVER['PHP_AUTH_USER']) )

{

if (isset($_SERVER['HTTP_AUTHORIZATION']) && (strlen($_SERVER['HTTP_AUTHORIZATION']) > 0)){

list($_SERVER['PHP_AUTH_USER'], $_SERVER['PHP_AUTH_PW']) = explode(':', base64_decode(substr($_SERVER['HTTP_AUTHORIZATION'], 6)));

if( strlen($_SERVER['PHP_AUTH_USER']) == 0 || strlen($_SERVER['PHP_AUTH_PW']) == 0 )

{

unset($_SERVER['PHP_AUTH_USER']);

unset($_SERVER['PHP_AUTH_PW']);

}

.htaccess

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L] RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ app.php [QSA,L]
</IfModule>

RewriteEngine On

RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L] RewriteCond %{REQUEST_FILENAME} !-f

RewriteRule ^(.*)$ app.php [QSA,L]

</IfModule>

14. May 2017

Inspired but not a copy – Underscore.php

Underscore.php features a good hundred of methods for all kinds of types : strings, objects, arrays, functions, integers, etc., and provides a parsing class that help switching from one type to the other mid-course.

Underscore.php

14. May 2017

Modern API to process – URI – in PHP

Simple and extensible API
RFC3986 compliant
Implements PSR-7 UriInterface interface
Framework-agnostic

use League\Uri\Parser;

$parser = new Parser();
var_export($parser('http://foo.com?@bar.com/'));
//returns the following array
//array(
//  'scheme' => 'http',
//  'user' => null,
//  'pass' => null,
//  'host' => 'foo.com',
//  'port' => null,
//  'path' => '',
//  'query' => '@bar.com/',
//  'fragment' => null,
//);

use League\Uri\Parser;

$parser = new Parser();

var_export($parser('http://foo.com?@bar.com/'));

//returns the following array

//array(

// 'scheme' => 'http',

// 'user' => null,

// 'pass' => null,

// 'host' => 'foo.com',

// 'port' => null,

// 'path' => '',

// 'query' => '@bar.com/',

// 'fragment' => null,

//);

URI