#django-nonrel

[marlinpaul]

Django Tutorial

Django is a web development framework that makes it easier to create and manage high-quality websites. Django makes the development process simple and time-saving by eliminating repeated operations. This Django tutorial will teach you all you need to know about Django.

Make sure you grasp the fundamentals of procedural and object-oriented programming, such as control structures, data structures and variables, classes, and objects, before moving on.

Django is a high-level Python web framework that promotes rapid development and simple, practical design. Django allows you to create better web apps faster and with less code.

History of Django

Adrian Holovaty and Simon Willison started it as an internal project at the Lawrence Journal-World newspaper in 2003.

In the year 2005, Django, named after jazz guitarist Django Reinhardt, was released in July 2005.

2005 Capable of handling many high-traffic sites.

Django is currently a global open source project with contributors from all over the world.

Django – Design Philosophies

The following design philosophies are included with Django:

Django is loosely coupled, which means that each part of its stack is independent of the others.

Less Coding Less code means faster development.

DRY stands for "Don't Repeat Yourself." Instead than repeating the same process over and over, everything should be developed only once.

Django's philosophy is to do everything it can to make development as quick as possible.

Django maintains a tight clean design across its own code, making it simple to follow standard web-development practises.

Advantages of Django

Here are a few benefits of utilising Django that can be found here:

Support for Object-Relational Mapping (ORM) Django acts as a link between the data model and the database engine, and it supports a wide range of databases such as MySQL, Oracle, and Postgres. Django's Django-nonrel branch also supports NoSQL databases. MongoDB and Google App Engine are the two NoSQL databases that are currently supported.

Django's built-in translation framework makes it possible to create multilingual websites. As a result, you can create a website that supports several languages.

Django comes with built-in support for Ajax, RSS, Caching, and a variety of other frameworks.

Django's administration GUI is a pleasant, ready-to-use user interface for administrative tasks.

Django comes with a lightweight web server that makes creation and testing of end-to-end applications a breeze.

This Django tutorial is for developers who want to learn how to create high-quality web apps with Django's clever strategies and tools.

[luckytaconinja]

ExcelR Data Science Courses In Chennai

The 9 Best Free On-line Information Science Programs In 2020

Machine studying is a valuable device to seek out patterns in massive information units. To handle giant knowledge units, data scientists must be familiar with databases. However, nonrelational databases are rising in reputation, so a greater understanding of database constructions is useful.

To Know More....

The course additionally contains a set of tangible skills that are the Django Python framework, Python IDE tools, Big Data, AWS, SAS, web optimization, Oracle, and Machine Learning. Few abilities that might be obtained upon completion of this Data Science course are such as Python, R, Tableau, Data Analysis, etc. Microsoft SQL fundamentals, transact sql, desk creation, SQL Server Indexes, Views, Stored procedures, triggers, T-SQL novices and superior, SSIS. Microsoft Azure, Data Lake, Data Factory, Azure PAAS, growing your applications on Azure, Migration of net sites to Azure, migrating .NET based net applications.

They need to collect sufficient knowledge to grasp the issue at hand and to higher remedy it in phrases of time, cash, and assets. This is a serious problem within the data-driven world that we live in at present. As a end result, most organizations are prepared to pay excessive salaries for professionals with applicable Data Science skills. IIM Kozhikode is NOT concerned in any means in the Career Services talked about right here. IIM Kozhikode or Eruditus do NOT promise or assure a job or development in your present job. Career Services is simply offered as a service that empowers you to handle your profession proactively.

Data Science Courses

In India, the typical Data Scientist wage can vary from Rs.1,000K to Rs.1,800K based mostly on expertise, location, and group. Mathematics plays a significant function in learning the methods of Data Science. Holding good mathematical experience would make your learning journey straightforward. Data Science predicts the future whereas Data Analytics day to day evaluation on the information.

BUSINESS NAME: ExcelR- Data Science, Data Analytics, Business Analytics Course Training Chennai, Address: Block-B,1st Floor, HansaBuilding RK SwamyCentre, 147, Pathari Rd, Thousand Lights, Chennai, Tamil Nadu 600006, phone: 085913 64838 email: enquiry@excelr.com

Data Science Courses

[timobook]

Python For Google App Engine

Python For Google App Engine: Master the full range of development features provided by Google App Engine to build and run scalable web applications in Python Download Introduction What this book covers  Chapter 1, Getting Started, will help you get your hands dirty with a very simple functional Python application running on a production server. The chapter begins with making a survey of Google's cloud infrastructure, showing where App Engine is placed and how it compares to other well-known cloud services. It then walks readers through downloading and installing the runtime for Linux, Windows, and OS X, coding a Hello, World! application and deploying it on App Engine. The last part introduces administration consoles both for the development and production servers. Chapter 2, A More Complex Application, teaches you how to implement a complex web application running on App Engine. It begins with an introduction to the bundled webapp2 framework and possible alternatives; then, you will get in touch with user authentication and form handling and then an introduction to Google's Datastore nonrelational database. The last part shows you how to make HTML pages through templates rendering and how to serve all the static files needed to style the page. Chapter 3, Storing and Processing Users' Data, will show you how to add more functionalities to the app from the previous chapter. The chapter begins by showing you how to let users upload files using Google Cloud Storage and how to manipulate such files when they contain image data with the Image API. It then introduces you to the task queues used to execute long jobs (such as image manipulation) outside the request process and how to schedule batches of such jobs. The last part shows you how to send and receive e-mails through the Mail API. Chapter 4, Improving Application Performance, begins by showing how to improve application performance using advanced features of Datastore. It then shows you how to use the cache provided by App Engine and how to break the application into smaller services using Modules. Chapter 5, Storing Data in Google Cloud SQL, is dedicated to the Google Cloud SQL service. It shows you how to create and manage a database instance and how to connect and perform queries. It then demonstrates how an App Engine application can save and retrieve data and how to use a local MySQL installation during development. Chapter 6, Using Channels to Implement a Real-time Application, shows you how to make our application real time, in other words, how to update what clients see without reloading the page in the browser. The first part shows how the Channel API works, what happens when a client connects, and what roundtrip of a message is from the server to the client. Then, it shows you how to add a real-time feature to our application from previous chapters. Chapter 7, Building an Application with Django, teaches you how to build an App Engine application using the Django web framework instead of webapp2. The first part shows you how to configure the local environment for development, and then the application from previous chapters is rewritten using some of the features provided by Django. The last part shows you how to deploy the application on a production server. Chapter 8, Exposing a REST API with Google Cloud Endpoints, shows you how to rewrite part of our application to expose data through a REST API. The first part explores all the operations needed to set up and configure a project and how to implement a couple of endpoints for our API. The last part shows explores how to add OAuth protection to the API endpoints. What you need for this book In order to run the code demonstrated in this book, you need a Python interpreter for version 2.7.x and the App Engine Python SDK as described in the Download and installation section from Chapter 1, Getting Started. Additionally, to access the example application once it runs on App Engine, you need a recent version of a web browser such as Google Chrome, Mozilla Firefox, Apple Safari, or Microsoft Internet Explorer. Who this book is for If you are a Python programmer who wants to apply your skills to write web applications using Google App Engine and Google Cloud Platform tools and services, this is the book for you. Solid Python programming knowledge is required as well as a basic understanding of the anatomy of a web application. Prior knowledge of Google App Engine is not assumed, nor is any experience with a similar tool required. By reading this book, you will become familiar with the functionalities provided by Google Cloud Platform with particular reference to Google App Engine, Google Cloud Storage, Google Cloud SQL, and Google Cloud Endpoints at the latest versions available at the time of writing this book. Via TimoBook

[eatdev]

Django-nonrel count() with limit on Google App Engine

Google's App Engine's has a Query.count(limit=<limit>)  method where the performance is based on the number of entities counted.  So the more entities in your database, the longer this takes to return.

You can short-circuit the count by including a limit, so even if there's a large number out there, the call will return within a somewhat-manageable time frame.

The problem is that Django's Queryset.count() method doesn't allow a limit parameter.  Luckily there's a hackaround:

queryset.query.high_mark = limit count = queryset.count() queryset.query.high_mark = None

With the default value of queryset.query.high_mark = None, it will run until it returns the full count, or potentially times out due to the large number of results.

[makkalot-blog]

Bu işlemi kolaylaştıran küçük bir blog yazısı yazdım.

[eatdev]

Running Django-nonrel in a shell on App Engine.

I'm not sure why setting PYTHONPATH doesn't work on my system.  Instead I have to:

import sys sys.path.append('') import djangoappengine.main

This sets up the shell so that it works like python manage.py shell.  At this point I can import my project's modules, including models for DB queries.  This also works for getting django set up inside App Engine's remote_api_shell.py

[eatdev]

Google App Engine Key Only queries on Django-nonrel

Traced through the Django-nonrel code to figure out how to do an App Engine key-only query, which is a bit more efficient at times, especially when you only want something like a count.

queryset = Model.objects.only('id')

Like any other queryset, you can append a filter to it, etc. This works well with my previous post about getting a limited count.

[eatdev]

Handling user ratings on App Engine

So far I've been running Django-nonrel on Google's App Engine.  This was due to familiarity with Django, and unfamiliarity with App Engine's Datastore.  I was able to go through about 2 months of work, with a fairly simple database, working pretty well with this setup.  Because the database was simple, I could move between App Engine and a SQL backend.

In the past week, I've refactored some code and added more advanced functionality.  I'm starting to run into Django limitations now.  First issue is pretty common.  App Engine's datastore doesn't support SQL JOINs.  I've worked around it with Django, but it's not a performance optimal version.  I haven't done a perfomance analysis, but maybe I'll do a writeup when I get to that.

I've run into a second issue with App Engine's write frequency limitation.  The proper way to get around it is to shard the item you're writing to.  There's an example from Google for updating a counter frequently.  I'm building a rating system, so I need to show an average value instead of a counter.  You only need to use add_to_average(name, value) on each new rating that's added, where name is a name of the counter (so you can use multiple independent counters), and value is the value you want to add to the average. You can retrieve the average value with get_average(name).

# Copyright ProjectEAT 2011 # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # Adapted from Google generalcounter.py # http://code.google.com/p/google-app-engine-samples/source/browse/trunk/sharded-counters/generalcounter.py from google.appengine.api import memcache from google.appengine.ext import db import random import logging class GeneralAverageShardConfig(db.Model): """Tracks the number of shards for each named average.""" name = db.StringProperty(required=True) num_shards = db.IntegerProperty(required=True, default=20) class GeneralAverageShard(db.Model): """Shards for each named average""" name = db.StringProperty(required=True) count = db.IntegerProperty(required=True, default=0) average = db.FloatProperty(required=True, default=0.0) def get_average(name): """Retrieve the value for a given sharded average. Parameters: name - The name of the counter """ average = memcache.get(name+"a") if average is None: sum = 0 divisor = 0 for subavg in GeneralAverageShard.all().filter('name = ', name): sum += subavg.average * subavg.count divisor += subavg.count average = sum / divisor logging.error("get_average from DB got average=" + str(average) + " divisor=" + str(divisor)) memcache.set_multi( { name+"a" : average, name+"c" : divisor }, time=60) return average def add_to_average(name, value): """Update and recalculate the value for a given sharded average. Parameters: name - The name of the average """ config = GeneralAverageShardConfig.get_or_insert(name, name=name) def txn(): index = random.randint(0, config.num_shards - 1) shard_name = name + str(index) average = GeneralAverageShard.get_by_key_name(shard_name) if average is None: average = GeneralAverageShard(key_name=shard_name, name=name) avgsum = average.average * average.count average.count += 1 average.average = (avgsum + float(value)) / average.count average.put() db.run_in_transaction(txn) # does nothing if the key does not exist client = memcache.Client() while True: # Retry loop mcavg = client.get_multi(["a","c"],key_prefix=name,for_cas=True) if len(mcavg) == 0: #Uninitialized average break logging.error("add_to_average from memcache got average=" + str(mcavg["a"]) + " divisor=" + str(mcavg["c"])) avgsum = float(mcavg["a"]) * float(mcavg["c"]) mcavg["c"] += 1 mcavg["a"] = (avgsum + float(value)) / mcavg["c"] retval = client.cas_multi(mcavg,key_prefix=name) if len(retval) == 0: break; def increase_shards(name, num): """Increase the number of shards for a given sharded counter. Will never decrease the number of shards. Parameters: name - The name of the counter num - How many shards to use """ config = GeneralAverageShardConfig.get_or_insert(name, name=name) def txn(): if config.num_shards < num: config.num_shards = num config.put() db.run_in_transaction(txn)

[makkalot-blog]

Sorunu çözen küçük bir blog girdisi yazdım.

[eatdev]

Using MapReduce with Django-nonrel on App Engine.

A while ago, I had read that the best way to clean up App Engine's datastore was to use the MapReduce API.  For one, you delete datastore entities in parallel.  Secondly, the datastore will return a maximum of 1000 entities per query, if you did it serially, you would have to loop through queries, potentially taking longer than the maximum execution time allowed for processing a single App Engine HTTP request.

Having changed my schema a bit, my Django app started failing when it loaded old data from my datastore.  I decided to try out the MapReduce to clean up some of the illegal old objects from my datastore.  I discovered that the MapReduce API doesn't work well with Django models.  It turns out the InputReader classes provided with the API is uses App Engine's python db API.  Fortunately, source is included, so I could write my own InputReader to map Django models instead of db models.

I left the API fetching entities, without converting them back to Django models.  This suited me well, since I was looking to fetch entities that wouldn't properly convert to my new Django models anyways.  Here's the code for the InputReader class.  I've tested it with App Engine SDK 1.6.2 (with the MapReduce bundle)

import djangoappengine.main from django.db.models.sql.query

import Query from mapreduce.input_readers

import AbstractDatastoreInputReader

from mapreduce import util from google.appengine.datastore import datastore_query

class DjangoKeyInputReader(AbstractDatastoreInputReader):

"""An input reader that takes a Django model ('app.models.Model') and yields Keys for that model"""

def _iter_key_range(self, k_range):

query = Query(util.for_name(self._entity_kind)).get_compiler(using="default").build_query()

raw_entity_kind = query.db_table query = k_range.make_ascending_datastore_query( raw_entity_kind, keys_only=True)

for key in query.Run( config=datastore_query.QueryOptions(batch_size=self._batch_size)):

yield key, key 

class DjangoEntityInputReader(AbstractDatastoreInputReader):

"""An input reader that takes a Django model ('app.models.Model') and yields entities for that model"""

def _iter_key_range(self, k_range):

query = Query(util.for_name(self._entity_kind)).get_compiler(using="default").build_query()

raw_entity_kind = query.db_table query = k_range.make_ascending_datastore_query( raw_entity_kind)

for entity in query.Run( config=datastore_query.QueryOptions(batch_size=self._batch_size)):

yield entity.key(), entity

The most time-consuming part of the project was trying to figure out the MapReduce API documentation. There's a few versions that come up when I do a google search. It turns out, it's all the same package, but the documentation comes from various dates.

The Mapper API is an older version that just covers the mapping portion of the pipeline.  This is what I used, and it still works.  It has an easy getting started guide.  The documentation, however, is outdated, yet is still contains details about the mapping portion that are missing in the newer documentation.  Ignore the old download which is still sitting around, use the latest MapReduce bundle which includes the old Mapper API.

The latest documentation covers the full MapReduce pipeline.  However, it just glazes over the entire pipeline at a high level, and isn't very useful for actual implementation.

What you want to download is the latest MapReduce bundle from the App Engine SDK download page. 

[tingletech]

The search order is same as that used by getattr() except that the type itself is skipped.

super(type[, object-or-type])

2. Built-in Functions — Python v2.7.1 documentation

[eatdev]

Django-nonrel and dev_appserver.py

When I first had Django-nonrel up and running, starting the local test server using either "python dev_appserver.py <project>" or "python manage.py runserver" (from inside the project folder would work equivalently.

At some point, the behavior changed, the two commandlines would both launch the test server, but the data sets that appeared were different.  Clearly they were using separate datastores, but I never really digged in to see what the difference was.  I just switched to using "python manage.py runserver", because the dumpdata and loaddata commands are pretty handy.

Today I was attempting to wipe the datastore, and ran into some roadblocks.  The commandline "python manage.py runserver --clear_datastore" didn't work.  The dev_appserver.py version "python dev_appserver.py --clear_datastore <project>" worked fine, but it only clears the dev_appserver.py datastore.

Turns out, using manage.py to launch the datastore uses the datastore in <project>/.gaedata/datastore, while using dev_appserver.py puts the datastore in /tmp/dev_appserver.datastore.  While I haven't gotten manage.py to work with --clear_datastore, you can user dev_appserver.py to clear your datastore by specifying the datastore path:

python dev_appserver.py --datastore_path=<project>/.gaedata/datastore --clear_datastore <project>

Benefit is now I can launch my projects using dev_appserver.py again. Oh, you can also just delete the datastore file.

[eatdev]

tastypie on django-nonrel on App Engine

There's a whole lot of documentation on the tastypie REST app for django, but surprisingly not much on getting it to work on Google App Engine.  I'm happy to say that it's quite easy to get it running on django-nonrel.

I installed django-nonrel into my GAE project, then simply copied tastypie in as a django-nonrel app.  It didn't work straight out of the box, but two very small code changes got it up and running.

Two changes tastypie/resources.py.  The first kills support for PATCH in tastypie, since it uses transactions, and they're not supported in django-nonrel.  I don't think I'm using PATCH yet, so I can live with this for the time being.

@@ -1858,8 +1858,10 @@ class ModelResource(Resource): Necessary because PATCH should be atomic (all-success or all-fail) and the only way to do this neatly is at the database level. """ - with transaction.commit_on_success(): - return super(ModelResource, self).patch_list(request, **kwargs) + assert 0, "patch_list unsupported due to transaction requirement!"

and the second change is the different way primary keys are done in django-nonrel.

@@ -1956,7 +1956,10 @@ class ModelResource(Resource): } if isinstance(bundle_or_obj, Bundle): - kwargs['pk'] = bundle_or_obj.obj.pk + try: + kwargs['pk'] = bundle_or_obj.obj.pk + except AttributeError: + kwargs['pk'] = bundle_or_obj.obj else: kwargs['pk'] = bundle_or_obj.id

And now I'm up and running.

[tingletech]

Need to do something like this for the A/V engine project to support repeating Foreign Keys.