Group by week mysql

How to group by week on mysql

Group by week in mysql is common specially for reporting. Here is how to do it.

If the column is with format yyyy-mm-dd


SELECT COUNT(*), YEARWEEK(__DATE_COLUMN__)
FROM __TABLE__ 
WHERE (__DATE_COLUMN_) > '2018-03-01 00:00:00'
GROUP BY YEARWEEK(__DATE_COLUMN__)

If the column is unixtime stamp format


SELECT COUNT(*), YEARWEEK(FROM_UNIXTIME(__DATE_COLUMN__))
FROM __TABLE__ 
WHERE FROM_UNIXTIME(__DATE_COLUMN__) > '2018-03-01 00:00:00'
GROUP BY YEARWEEK(FROM_UNIXTIME(__DATE_COLUMN__))

Where the __TABLE__ is he mysql table and __DATE_COLUMN__ is the column you are interested.

Add scrollbar to mysql result in terminal

MySQL command line scrollbar

MySQL command line provides a full fledged application that allows interaction with mysql engine so easy.

In MySQL command one thing that would annoy is when there is larger result set and scrolling is static or not even accessible.

If you have larger result coming from mysql and you want to scroll through the result on mysql terminal?

You might ask

How to add scrollbar to mysql result in mysql command line?

Reading larger result set in mysql command

Or have you asked how to scroll on results of terminal mysql?

Solution

Go to your terminal and do


mysql> pager less

This would allow you to have that feeling of being on vim

See how to configure LAMP on Amazon here

mysql command not found on mac after installation

After downloading mysql dmg on mac if you go to terminal and try to issue mysql command and get mysql command not found there keep reading.

This, in most cases, is the terminal not knowing the mysql command unless you go to the actual installation folder.

To check that out, go to

/user/local/mysql/bin/mysql --version

If this is giving you the right mysql information, then it is a matter of telling the terminal what the mysql command is.

Temporary fix

export PATH=$PATH:/usr/local/mysql/bin

The above command would allow the command files to be recognized by the current issuing terminal. But you won’t be able to access those on new terminal

Permanent fix

echo "export PATH=$PATH:/usr/local/mysql/bin" >>~/.bash_profile

The above command would simply append the path command you exported to the .bash_profile file.

Screen Shot 2016-04-10 at 9.21.59 PM

Upgrade mysql version from 5.5 to 5.6

Upgrade mysql version

This would probably make sense on Amazon EC2 instance where it comes by default with
mysql 5.5, as of this writing, and you want to upgrade mysql to the next version.

In this example I have used version 5.5 for the upgrade mysql example. But, technically the methodology would work for other versions as well unless mysql drastically changes folder locations.

How to upgrade mysql from 5.5 to 5.6

First go ahead and move the mysql 5.5 to new folder

sudo mv /var/lib/mysql /var/lib/mysql55

Then grab the tar from mysql download page [Here]

Once you download the tar file containing the new mysql version do the following commands on your terminal.

sudo tar -xvf yourDownloadedTar.tar
sudo rpm -ivh MySQL-shared-5.6.17-1.el6.x86_64.rpm
sudo rpm -ivh MySQL-client-5.6.17-1.el6.x86_64.rpm
sudo rpm -ivh MySQL-server-5.6.17-1.el6.x86_64.rpm

This would upgrade your current mysql version to mysql 5.6 or which ever is the new mysql version.

Then restart mysql for the change to take effect.

Configuring PHP + MySQL + Apache on Amazon EC2 Step by Step

Log in to aws.amazon.com
Click on EC2 virtual servers in the cloud

From the left menu, under Network and Security, select Key Pairs
Create on by hitting “Create Key Pair”. Mind you, aws will give you only one chance to save the pem file you will be using for logging. So make sure you download and save it.

From your select Elastic Beanstalk and follow the wizard to create an instance per your need.
in the wizard, there is a step you will be asked to use the pem file you downloaded.

Now from the left menu, click on the instances.
And click on the instance on the right pane and you will see another pane with description will be opened on the lower part. On that pane, search for security group and click the link
You will be transferred to the security group associated with that instance. Click on the “in bound” tab and check if SSH is listed there, if not, hit the ‘Edit’ button and add a new rule of SSH with the source of anywhere if you would like to ssh into your box from anywhere or you can specify particular ip address.

log to your instance from terminal as

ssh -i /your/downloaded/pem/file ec2-user@public-domain-goes-here

you will get your public domain on the instance you selected.

If you are using Elastic Beanstalk, it will come with installed apache server for as your webserver. Just restart it

sudo service httpd restart

MySQL shell would be there as well but not the mysql server so install that

sudo yum install -y mysql-server

And restart the demon

sudo service mysqld restart

I have checked if git is installed, if you are using any DCVS, which you should and it is installed already to verify do

git --version

Next would be the creation of your public and private keys for secure communication with the other servers.

ssh-agent -t rsa -b 4096 "your_email@domain.tld"

The above command will provide you with the public and private key that you would use. The default path for it would be on ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub

If you are using github or bitbucket, you would need the content of the id_rsa.pub for logging to the server without password.

If you are going to use mongo just use the very information on the following links https://docs.mongodb.org/ecosystem/platforms/amazon-ec2/
http://www.liquidweb.com/kb/how-to-install-the-mongodb-php-driver-extension-on-centos-6/

This will get you started with your aws instance

Access MySQL from Host being on Vagrant

For this all you would need is tunneling/port forwarding from vagrant to mysql.

Being on vagrant machine, figure out what the host ip is, usually netstat would tell you that.
The default being 10.0.2.2 and assuming you have 3306 port on host machine do the following

ssh username@10.0.2.2 -L 3306:127.0.0.1:3306 -N

The above would forward the 3306 [ the one on the left ] to the hosts port. Mind you, you have to use 127.0.0.1 to access your vagrant machine in place of localhost which might not work.

Setting new Vagrant machine for PHP-Mysql development

As a developer you would know how stressful it would be to setup a machine. Even once you figure this out and set your machine working, you might be surprised by how it would get wacky easily when you update OS for specific packages. This is specially true mostly for *nix boxes.
Also this holds true for the new developer joining the team. Someone has to help him and all the time something might now work b/c of newer version of OS or something else.

The solution? Vagrant
Vagrant is relatively the new kid in this area. In short, it is like having a computer in your computer that is preconfigured with all what you need so that you can just start coding without having the hustle of installing 40+ packages.

Lets have PHP+MySQL+Memcached+.. setup using Vagrant

1. First thing first Get the provider
I am assuming the dev machine in this tutorial so the provider would be VirtualMachine
Get your share from Here

2. Install vagrant from Here

Go through the above links, download them and install them.. I am sure you rock star on this one :)

3. Start the provision. The advanced way to have provision would be either through the puppet or chef files.
But, for the php there is awesome site for simulating this. Go to Here and invest sometime there..
For now, just follow the wizard there until you get the box, we will replace the config file anyway.

4. Once you got your zip file unzip it wherever you want and replace the config file with the following

---
vagrantfile-local:
    vm:
        box: puphpet/ubuntu1404-x64
        box_url: puphpet/ubuntu1404-x64
        hostname: 'vagrant-local'
        memory: '1024'
        cpus: '1'
        chosen_provider: virtualbox
        network:
            private_network: 192.168.56.101
            forwarded_port:
                BXfJQibZ50h1:
                    host: '8640'
                    guest: '22'
                mysql:
                    host: '3307'
                    guest: '3306'
        post_up_message: ''
        provider:
            virtualbox:
                modifyvm:
                    natdnshostresolver1: on
            vmware:
                numvcpus: 1
            parallels:
                cpus: 1
        provision:
            puppet:
                manifests_path: puphpet/puppet
                manifest_file: site.pp
                module_path: puphpet/puppet/modules
                options:
                    - '--verbose'
                    - '--hiera_config /vagrant/puphpet/puppet/hiera.yaml'
                    - '--parser future'
        synced_folder:
            K2NewVabIO4S:
                owner: www-data
                group: www-data
                source: ./
                target: /var/www
                sync_type: nfs
                rsync:
                    args:
                        - '--verbose'
                        - '--archive'
                        - '-z'
                    exclude:
                        - .vagrant/
                    auto: 'false'
        usable_port_range:
            start: 10200
            stop: 10500
    ssh:
        host: null
        port: null
        private_key_path: null
        username: vagrant
        guest_port: null
        keep_alive: true
        forward_agent: false
        forward_x11: false
        shell: 'bash -l'
    vagrant:
        host: detect
server:
    install: '1'
    packages:
        - build-essential
        - memcached
        - vim
        - curl
        - git-core
        - imagemagick
        - sendmail
        - libmemcached-dev
        - htop
        - unzip
        - mercurial  
users_groups:
    install: '1'
    groups: {  }
    users: {  }
cron:
    install: '1'
    jobs: {  }
firewall:
    install: '1'
    rules: null
apache:
    install: '1'
    settings:
        user: www-data
        group: www-data
        default_vhost: true
        manage_user: false
        manage_group: false
        sendfile: 0
    modules:
        - rewrite
    vhosts:
        ekjd369uxkx8:
            servername: my-local-website.com
            serveraliases:
                - www.my-local-website.com
            docroot: /var/www/local-website
            port: '80'
            setenv:
                - 'APP_ENV dev'
            directories:
                twe3plnvcymi:
                    provider: directory
                    path: /var/www/local-website
                    options:
                        - Indexes
                        - FollowSymlinks
                        - MultiViews
                    allow_override:
                        - All
                    require:
                        - all
                        - granted
                    custom_fragment: ''
            engine: php
            custom_fragment: ''
            ssl_cert: ''
            ssl_key: ''
            ssl_chain: ''
            ssl_certs_dir: ''
    mod_pagespeed: 0
nginx:
    install: '0'
    settings:
        default_vhost: 1
        proxy_buffer_size: 128k
        proxy_buffers: '4 256k'
    upstreams: {  }
    vhosts:
        ayer2gfhqh9a:
            proxy: ''
            server_name: my-local-website.com
            server_aliases:
                - www.my-local-website.com
            www_root: /var/www/local-website
            listen_port: '80'
            location: .php$
            index_files:
                - index.html
                - index.htm
                - index.php
            envvars:
                - 'APP_ENV dev'
            engine: php
            client_max_body_size: 1m
            ssl_cert: ''
            ssl_key: ''
php:
    install: '1'
    version: '55'
    composer: '1'
    composer_home: ''
    modules:
        php:
            - cli
            - intl
            - mcrypt
            - curl
            - memcached
            - sqlite
        pear: {  }
        pecl:
            - pecl_http
    ini:
        display_errors: On
        error_reporting: 'E_ALL & ~E_STRICT'
        session.save_path: /var/lib/php/session
    timezone: America/Los_Angeles
    mod_php: 0
hhvm:
    install: '0'
    nightly: 0
    composer: '1'
    composer_home: ''
    settings:
        host: 127.0.0.1
        port: '9000'
    ini:
        display_errors: On
        error_reporting: '-1'
    timezone: null
xdebug:
    install: '1'
    settings:
        xdebug.default_enable: '1'
        xdebug.remote_autostart: '0'
        xdebug.remote_connect_back: '1'
        xdebug.remote_enable: '1'
        xdebug.remote_handler: dbgp
        xdebug.remote_port: '9000'
xhprof:
    install: '0'
wpcli:
    install: '0'
    version: v0.17.1
drush:
    install: '0'
    version: 6.3.0
ruby:
    install: '1'
    versions:
        t4DlEFgKoFSk:
            version: ''
nodejs:
    install: '1'
    npm_packages: {  }
python:
    install: '1'
    packages: {  }
    versions:
        juTtbSrrnLMG:
            version: ''
mysql:
    install: '1'
    override_options: {  }
    root_password: '123'
    adminer: 0
    databases:
        YET9UxaCD4KV:
            grant:
                - ALL
            name: dbname
            host: localhost
            user: dbuser
            password: '123'
            sql_file: ''
postgresql:
    install: '0'
    settings:
        root_password: '123'
        user_group: postgres
        encoding: UTF8
        version: '9.3'
    databases: {  }
    adminer: 0
mariadb:
    install: '0'
    override_options: {  }
    root_password: '123'
    adminer: 0
    databases: {  }
    version: '10.0'
sqlite:
    install: '1'
    adminer: 0
    databases: {  }
mongodb:
    install: '1'
    settings:
        auth: 1
        port: '27017'
    databases: {  }
redis:
    install: '1'
    settings:
        conf_port: '6379'
mailcatcher:
    install: '1'
    settings:
        smtp_ip: 0.0.0.0
        smtp_port: 1025
        http_ip: 0.0.0.0
        http_port: '1080'
        mailcatcher_path: /usr/local/rvm/wrappers/default
        from_email_method: inline
beanstalkd:
    install: '0'
    settings:
        listenaddress: 0.0.0.0
        listenport: '13000'
        maxjobsize: '65535'
        maxconnections: '1024'
        binlogdir: /var/lib/beanstalkd/binlog
        binlogfsync: null
        binlogsize: '10485760'
    beanstalk_console: 0
    binlogdir: /var/lib/beanstalkd/binlog
rabbitmq:
    install: '1'
    settings:
        port: '5672'
elastic_search:
    install: '1'
    settings:
        version: 1.4.1
        java_install: true
solr:
    install: '1'
    settings:
        version: 4.10.2
        port: '8984'
php_memcached:
    install: '1'

** I have used 1G memory size for the guest. You can increase or decrease based on your physical memory accounting.

**Notice the usage of nfs here. In the case of windows OS, it might not work as expected so you might have to select samba or something to for the file synching part. I haven’t tried anything on windows for a while so don’t quote me on this.

As you might have guessed it, this is the heart of the provision. This will telling your little box what it should stuff to itself to be ready for PHP coding.
4. Now you have all the necessary stuff to get started. So far your machine has php, apache2 web server, mamcached and radis for your caching, mongodb incase you need it, git and mercurial for your versioning and more server goodies.
So if you want to add server based packages, go under the server and add your package. In this provision, ubuntu distro is used. There are many flavors out there. So, when you add the server package make sure the appropriate bundle is sited there to get away from surprises.

Being on the same directory as your config.yaml file, run

vagrant up

This one liner, will read the config file and populate your box with all the packages you listed with web server and and everything.

** If you see something strange like a bunch of red lines, please read it carefully and try to research it. You can also ask here and I will try to help.

Hopefully, it will all run nicely

5. Now you can say hi to your new machine by logging into it using

vagrant ssh

The above command will land you the virtual machine packed with all the software you have requested.. Feel free to move around, check apache, memacache and the like..

6. Now, if you notice, there is a part on the config.yaml file that mentions local-website
Lets talk a bit about it.
Vagrant will sync the folders between the host and the guest boxes. That is a great relief. You can work on any files from your favorite IDE on your host and all the changes would take effect on the guest machine – the trick is you are updating the same file. Thanks to nfs.
So, Vagrant will sync anything on the host directory that hosts the Vagrantfile to the guest sync folder in accordance to how you tell it. We have told it to match anything in the aforementioned folder to be syncd with /var/www inside the guest machine. So if you go to your /var/www folder inside your guest box, you will see the replica of the files and directories there.

Create a folder local-website inside the directory where you unzip the zip file, that is the one holding the Vagrantfile file.

mkdir local-website

And create an index file inside it

touch local-website/index.php

Inside the index.php just put the phpinfo()

<?php
phpinfo();

7. Moment of truth!
We will try to access the web from our guest machine being on the host machine.
Before that, we have tell to our host machine what my-local-website.com mean. See how this is used as server name in the config.yaml file

Go to your host’s /etc/hosts file and insert 192.168.56.101 my-local-website.com

sudo echo "192.168.56.101 my-local-website.com" >> /etc/hosts

What this does is, when you type my-local-website.com on your browser, before it is relayed to the outer DNS, it will check if it has to do something with it. In this case it will forward it to 192.168.56.101.
If you lookup for this ip-address in the config.yaml file we created, you will see it under the network. This is the ip-assigned for the host in the little host-guest network vagrant provides.

Now you can go to your favorite browser and type my-local-website.com to see the phpinfo() of your guest machine’s php settings.

This will end up the journey. Now you have a fully working box that you can save it somewhere and reuse it anytime.

8 Bonus.
To shutdown the box use

vagrant halt

To destroy the box – this will, as the name implies remove everything

vagrant destroy

If you update the config file, you have to re-provision it by

vagrant provision

error duplicate entry for key when creating new unique key mysql

If you got the above message when you try to add a new unique key on multiple column in mysql as

ALTER TABLE table_name ADD UNIQUE (col1, col2, col3);

Mysql would alert you if there is an already existing row in your table that is violating the newly added constraint.
Say you want to have uniqueness on col1 and col2 and assume col1 has value of “one” and col2 has value of “love”

Before this constraint you can have another row with the values of “one” and “love” for col1 and col2 respectively. But now that is not possible so you have to take care of that before applying

database

Connecting to Vagrant [PuPHPet] MySql from host

Probably here with – I can’t connect to mysql on vagrant machine Yup I have been there

If you are php developer and are using vagrant you would want to use mysql tool to access the vagrant mysql server.

Here is the step by step procedure.

I will assume the PuPHPet configuration
First, get the user which can connect from any host

Log into your vagrant

vagrant ssh

Then open the mysql configuration file

vi /etc/mysql/my.cnf

And update the bind-address part to be 0.0.0.0. This will allow address to be used from any host.
If you want to specifically access from some host only, then put IP over there.

Then being in your vagrant machine run

mysql -uroot -p123

This will be depending on if you have provided a password for root the default on puPHPet would ask password so you would have a password

After this allow root user to access everything from everywhere.. or you can create another user and grant that

mysql > GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123';
mysql > FLUSH PRIVILEGES;

Then to make the sql pickup the changes made so far

sudo service mysql restart

Then being on your host machine open your sequel pro – if you are on mac.. or other clients you would use for mysql.

Mysql Host: 127.0.0.1
Mysql user: root
Mysql Password: 123

SSH Host: 127.0.0.1
SSH Username: vagrant – this is the default one
SSH key -> point the key in puphpet/files/dot/ssh/id_rsa
SSH Port: 2222 – this is also the default ssh port forwarding, if you change this change it here too..

See how to Setting up vegrant machine from scratch here