How can i improve performance of mysql table which has 1600000 rows?

how can i improve performance of mysql table which has 1600000 rows? by what value should i change mysql variables? so that i gain performance?

mysql> SHOW CREATE TABLE gps_device_data_demo\G; *************************** 1. row *************************** Table: gps_device_data_demo Create Table: CREATE TABLE `gps_device_data_demo` ( `imei_no` bigint(15) NOT NULL DEFAULT '0', `ioelementpriority` int(1) DEFAULT NULL, `longitude` int(15) DEFAULT NULL, `latitude` int(15) DEFAULT NULL, `speed` tinyint(3) unsigned DEFAULT NULL, `iovalue` varchar(254) CHARACTER SET latin1 DEFAULT NULL, `odom` bigint(20) DEFAULT NULL, `alarm_category_realtime_id` varchar(70) CHARACTER SET latin1 DEFAULT NULL, `angle` int(4) DEFAULT NULL, `altitude` int(4) DEFAULT NULL, `satellites` int(4) DEFAULT NULL, `data_received_time` bigint(20) NOT NULL DEFAULT '0', `continue_data_received_time` bigint(20) NOT NULL DEFAULT '0', `movement` int(2) DEFAULT NULL, `eventsource` int(4) DEFAULT NULL, `data_location` varchar(250) CHARACTER SET latin1 DEFAULT NULL, `data_poi` varchar(100) CHARACTER SET latin1 DEFAULT NULL, `data_validity` enum('valid','invalid') CHARACTER SET latin1 DEFAULT NULL, `createdby` varchar(10) CHARACTER SET latin1 DEFAULT NULL, `createddate` datetime DEFAULT NULL, `modifiedby` varchar(10) CHARACTER SET latin1 DEFAULT NULL, `modifieddate` datetime DEFAULT NULL, `deletedby` varchar(10) CHARACTER SET latin1 DEFAULT NULL, `deleteddate` datetime DEFAULT NULL, PRIMARY KEY (`imei_no`,`data_received_time`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8

How can i improve performance of mysql table which has 1600000 rows?

How can i improve performance of mysql table which has 1600000 rows?

how can i improve performance of mysql table which has 1600000 rows? by what value should i change mysql variables? so that i gain performance?

mysql> SHOW CREATE TABLE gps_device_data_demo\G; *************************** 1. row *************************** Table: gps_device_data_demo Create Table: CREATE TABLE `gps_device_data_demo` ( `imei_no` bigint(15) NOT NULL DEFAULT '0', `ioelementpriority` int(1) DEFAULT NULL, `longitude` int(15) DEFAULT NULL, `latitude` int(15) DEFAULT NULL, `speed` tinyint(3) unsigned DEFAULT NULL, `iovalue` varchar(254) CHARACTER SET latin1 DEFAULT NULL, `odom` bigint(20) DEFAULT NULL, `alarm_category_realtime_id` varchar(70) CHARACTER SET latin1 DEFAULT NULL, `angle` int(4) DEFAULT NULL, `altitude` int(4) DEFAULT NULL, `satellites` int(4) DEFAULT NULL, `data_received_time` bigint(20) NOT NULL DEFAULT '0', `continue_data_received_time` bigint(20) NOT NULL DEFAULT '0', `movement` int(2) DEFAULT NULL, `eventsource` int(4) DEFAULT NULL, `data_location` varchar(250) CHARACTER SET latin1 DEFAULT NULL, `data_poi` varchar(100) CHARACTER SET latin1 DEFAULT NULL, `data_validity` enum('valid','invalid') CHARACTER SET latin1 DEFAULT NULL, `createdby` varchar(10) CHARACTER SET latin1 DEFAULT NULL, `createddate` datetime DEFAULT NULL, `modifiedby` varchar(10) CHARACTER SET latin1 DEFAULT NULL, `modifieddate` datetime DEFAULT NULL, `deletedby` varchar(10) CHARACTER SET latin1 DEFAULT NULL, `deleteddate` datetime DEFAULT NULL, PRIMARY KEY (`imei_no`,`data_received_time`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8

How can i improve performance of mysql table which has 1600000 rows?

JADE in NodeJS Tech Stack

I am working on a POC on Node JS, and I learnt that a typical tech stack will look like – Jade (instead of HTML)/ NodeJS/ and some database. My question instead of Jade can we use HTML 5? This is to avoid learning one more language to complete the POC. Also I assume that I will be able to expose the Node JS methods as rest API instead of having PHP or Java layer.

More over if I use simple HTMl/JQuery – for UI and Node.js ( for restful service) it will be easy for one to migrate to other framework easily. Please share your experience.

read more

JADE in NodeJS Tech Stack

I am working on a POC on Node JS, and I learnt that a typical tech stack will look like – Jade (instead of HTML)/ NodeJS/ and some database. My question instead of Jade can we use HTML 5? This is to avoid learning one more language to complete the POC. Also I assume that I will be able to expose the Node JS methods as rest API instead of having PHP or Java layer.

More over if I use simple HTMl/JQuery – for UI and Node.js ( for restful service) it will be easy for one to migrate to other framework easily. Please share your experience.

read more

store data in 2d-scipy array

I am trying to store data in an array/list. But, due to my limited experience in python, can’t do that. As you can see, currently, I can at best write them to a file, but for post-processing, I need to read them in numpy/scipy array.
How I can do that?

for subdir, dirs, files in os.walk(sys.argv[1]):
    for inp in files:
        if inp.endswith("_SCF.out"):
            fsys = subdir + "/" + inp
            with open(fsys) as finp:
                for line in finp:
                    if "lattice constant  ALAT" in line:
                        lata = float(line.strip()[-7:])/1.88973
                    if " ERR " in line:
                        mom = line.strip()[61:70]
                    if "SCF - cycle converged" in line:
                        etot = float(line.lstrip()[10:25])
                        of.write("{:<10f} {:<15f} {:<15f}\n"
                                 .format(lata,  etot, float(mom)))

read more